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ABSTRACT 

Field ^tests on- an interpersonal skills 
program, the Heartsmart Adventures, were^ conducted for 
and summative evaluation in 14 classrooms in 7 schools 
students in grade 4 classes •> The results indicate that 
program is likely to provide an appealing, educationally relevant and 
worthwhile experience to fourth grade children regardless of 
ethnic/racial or socio-economic backgrounds; the program as a whole 
can be easily administered by the teacher, although certain specific 
activities require special group process skills to achieve maximum 
results; and the program -will teach fourth grade students to be more 
proficient in identifying feelings and inferring feelings from the 
behavior of others, identifying ways in which people behave when they 
have certain feelings, determining f eelings.'^in others by psking, 
expressing needs and desires and taking appropriate actions to 
accomplish them, and expressing feelingsT" Current weaknesses o% the 
program were found to be related to sex and racial/c^thnic bias in the 
program materials, and to the iback of sufficient and clearly 
presented direction to teachers about the program goals and teaching 
methods. Extensive revisions of the program materials and teacher's 
manual were documented in the report to suggest how these two major 
weaknesses of the program were remedied. (PN) 
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ABSTRACT 

In the spring of 1977 a National Inj?titute of Edu n supported 
field' test) evaluation study was conducted by Research i:^. etter Schools 
on ^an interpersonal skills instructional program known as The Heartsmart 
Adventures. The goals of the field test were to conduct both formative 
and summative evaluation of the program. 

The field test was conducted in 14 classrooms in seven schoo°ls in 
or near Philadelphia, Pennsylvania. The sample consisted of 394 stu- 
dents in 4th grade classes, and included- students of both sexes and 
most ethnic groups. Half the classrooms recieved instruction in the 
« Heartsmart program, and the other half served as comparison classes. 

Two R3S developed criterion-referenced tests, the What Would You 
Do? test and the What's Happening? test, and one standardized measure, 
the My Class Inventory , were used to^ffesess program outcomes in relation 
to the program's instructional goals. The children's version of the 
Fundamental Interpersonal Relations Orientation Scales FIRO-BC) ,x 

developed by William ■ Schutz , was used to explore pcfssible program .im- 
pacts on students' interpersonal needs. Data obtained by pre- to post- 
testing of the above measures were analyzed primarily by repeated mea- 
sures design multivariate analysis of variance procedures. Formative 
evaluation was largely conducted by "consultant review," observational,, 
and interview procedures. 

A summary of conclusions based on evaluation findings follows. 
The field test version of Heartsmart may be perceived as relatively 



harmless, although certain situations, elements or concepts of t:be pro- 
gram were found to be potentially harmful, and directions to the teaoher 
allowed t(5t) much latitude for potentially harmful variations. While 
social fairness in the materials may have been achieved. in quantitative 

terms, a pervasive sex and racial/ethnic bias was present in the per- 

•-3 

trayal of female and non-white characters. The program was implemented 
as intended at all sites, and was not distorted by variations or innor- 
vations. The program was found to be appealing to at least 75% of the 
"'students, and teachers considered the program both appealing and educa- 
tionally worthwhile. The evidence indicated that almost all Heartsmart 
pupils grasped the main ideas presented in tape-led leesons, although 
other factors besic'es Heartsw^rt instruction ifiay have accounted for 
s::udent achievement results. Conclusions were mixed with riespec t to 
educational effectiveness. The Heartjmart program was judged to be^ 
successful in achieving the objective of teaching students several spe- 
cific interpersonal skills, and possibly of facilitating the applica-- 
tion of these skills to out-of-class peer and adult interpersonal situ- 
ations. No definitive conclusion was reached with respect to the pro- 
gram's ability to,- improve the perceived climate of the classroom. The 
quality and quantity o^ evidence presented in reference to evaluating 
the program's ability co produce a better match between students' ex- 
pressed and wanted interpersoi^al needs in the areas of inclusion and 
affection behavior were insufficient to reach a definitive conclusion. 
The Heartsmart program may have produced 'the desired effects for girls 
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to a greater extent than for boys. Pre-existing social climate differ- 
ences in the classrooms did not measurably effect the nature or extent 
of program effects. Program outcomes did not appear to be related Go 
implementation variations, 

rhe major strengths of the Hea'rtsmart program were judged to be 
as follows: the Heartsmart program is likely to provide an appealing, 
educationally relevant and worthwhile experience to fourth grade chil- 
dren regardless of eunnic/racial or. socio-economic backgrounds; 
the program as a whole can be easily administered by the teacher, al- 
though certain spe.^if ic activities require special group process skills 
to achieve maximum results; and the program will teach fourth grad^ stu- 
dent > to be more proficient in identifying feelings and inferring feel- 
ings from the behavior oi others, identi'fying ways in which people be^ 
have when they have' certain feelings, determining feelings in others by 
asking, expressing needs and desires and taking ^appropriate actions to 
accomplish them, and expressing feelings. Current weaknesses of the 
program were found to be related to sex and racial/ethnic bias in the 
program materials, and to the lack of sufficient and clearly presented, 
direction to teachers about the program goals and teaching methods, 
Extensive revisions of the program materials and teacher's manual were 
documented in the report to suggest how two major weaknesses of the 
program were remedied. 
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CHAPTER^ ONE: INTRODUCTION 

This chapter presents background information concerning previous 
related evaluation work, delineates major audiences for, this repcrt, 
and describes the content "and organization of the report. 
Background 

Since 1974 Research for Better Schools (RBS), through support pro- 
vided by the National Institute of Education, has been engaged in con- , 
ceptualizing, designing and developing an interpersonal skills instruc- 
tional course known as The Heartsmart Adventures. This program, initi- 
ally labeled Testing Interpersonal Hypotheses (TIH) , was 'designed to 
teach 9-11 year old children important concepts and skills that would 
enable' them to participate more effectively in their interpersonal 
relationships. 

Evaluation ha'5 played an essential, and continuous r :le in the 
development of this program. During 1974-75, prototype materials of 
individual program . components were tested and a tryout of the entire 
program was conducted in a single, racially mixed fourth grade class- 
room in Los Angeles (Berzon, _et al. , 1975).' This preliminary evalua- 
tion focused on students' ability to follow the storyline of the pro- 
gram and comprehend th^e basic concepts presented in the materials. The 
extent to which the lesson materials were appealing and attention- 
holding was also a primary question addressed in this early evaluation. 
Evaluation results from initial tryouts of the program materials were 
used to make revisions. The revised program was then subject co a 

1 
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pilot test during 1975-76 in which the major concerns were to gather 
preliminary summative evaluation as well as additional formative evalu- 
ation data; the pilot test sample included students in eight fourth 

r 

grade classrooms in Los Angeles, Information about program appeal, 
student learning, program effectiveness relative to instructional objec- 
tives, teacher implementation, relevance and usefulness of the program 
for low SES, Black minority children, and parental reaction was gathered 
during the pilot test. Based on evaluation results (Berzon, et al, , 
1976), a second set of major revisions was made in the program. To com- 
plete the development/evaluation cycle for the program, a field test - 
was conducted during 1976-77 in the Philadelphia, Pa, metropolitan area 
mainly for the purpose of assessing the program's effectiveness to de- 
liver on its goals and objectives, but also for the purpose of complet- 
ing materials revisions in anticipation of marketinj^ the program for 
commercial use in 1978. The content of this report relates principally 
to the methods and results of this field test. 
Audiences 

The major audience for this report is the National Institute of 
Education, as the sponsor for the development of the Heartsmart program. 
It is worth noting that this report may be valuable to prospective 
school system personnel interested in adopting this program within . 
their elementary school curriculum. Another possible audience for this 
report, or part? herein, is the Joint Office of Education/National 
Institute of Education Dissemination Review Panel. The major responsi- 
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CHAPTER TWO: PROGRAM DESCRIPTION 



The description of the program presented here relates to that used 
in the field test, and does not reflect or include revisions made, as a 
result of ds^g collected during the field test* 

I. PROGRAM GOALS 

o 

The Heartsmart Adventures is a program for students, ages nine to 
eleven, designed to enable them to develop constructive human relation- 
ships, ^ 

The program consists of thirty lessons divided into three Units. 

The twelve lessons of Unit I are designed to enable students to: 

• experience themselves in the world more fully through 
increased awareness of their senses and feelings; 

• express their emotional needs „more proficiently and 
with increased versatility; 

• respond more effectively to others through improved 
understanding of the reciprocal relationship between 
their own and others' feelings and behaviors; 

• cooperate more effectively with others; 

• transfer interpersonal skills learned in one situation 
to other situations. (Berzon, 1977), 

r 

The nine lessons of Unit II are designed o that 

students — through peer group interaction - — will be able 
to identify and express their individual needs to establish 
and maintain satisfactory relationships with others with 
respect to: 

• interaction and inclusion; 

• power and control; 

• love and affection. (Berzon, 1977). 

These three affective areas, inclusion, control and affection* 
''constitute a sufficient set of areas of interpersonal behavior for the 
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prediction and* explaniation of interpersonal phenomena." (Schutz, 1966).' 

The nine lessons of Unit III are designed to 

enable students to identify and express their individual 
needs to feel that they are: 

• significant and worthwhile human beings (inclusion); 

• competent, responsible persons (control); 

• lovable individuals (affection) (Berzon, 1977). 

II. CONTENT AND CCMPONENTS 



The program consists of thirty lessons, each designed to take 
approximately half an hour. Twenty-four of the lessons are tape-led. 

Six lessons are discussions, called Heart to Heart talks, led by the 

\ 

teacher. In addition to these lessons there are optional enrichment 
activities to be used at the teacher's discretion. 



The tapes tell the dramatized story of a boy named Jack, from the 

\ 

Kingdom of Hearts, who has many adventures as he finds his feelings 
and learns to express them. In the story Jack learns, among other 
things, to understand himself and other people better. 

The tapes also guide students in individual and small group acti- 
vities for which they use gamepages and/or program accessories. 

The six teacher-led Heart to Heart talks interspersed throughout 
the program, are designed to provide students with the opportunity to 
discuss ways in which they relate program concepts to their own personal 
experiences . 

The program is divided into three units. Unit I, introducing the 
language, concep ts , and storyline, requires that students work individu-- 
ally. In Units II and III students work in six-person groups. 
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Materials 

Each class which participated in the field test received the fol- 
lowing materials: 

• A teacher's manual containing le.;£;son notes, story and tape-led acti- 
vity summaries, illustrations of student materials, suggestions and 
masters of worksheets for optional activities, discussion questions 
for the Heart to Heart talks, and guidelines and descriptions of 
program concepts and components, 

• Student Journal pages for each student for each lesson, consisting of 
strip-cdrtoon__Storypages (to -be colored while s tudents-listen-tO; the — 

story), Gamepages (to be completed for the tape-led activities), and 
Reviewpages (consisting of questions about the lessens (s) reviewed on 
the tape) • - * 

• A song book of the 8 Heartsmart songs for each student, 

• A set of tape cassettes consisting of 24 lesson tapes, 2 song tapes, 
and a leadership tape, (The latter is designed to teach students elec 
ted as small group leaders" to manage group tasks effectively.) 

• Small accessories for each small group, consisting of 1 cardboard 
microphone, 2 TV screens, 6 yarn loops (color coded), 6 picture pages, 
and 2 host pages, 

e Large accessories for each small group, consisting of a Group Journal 
(a desk-top board 22" X 44"), a large picture to color, and a "slice" 
of a paper "pie" to color. 

All accessories were used for tape-led activities. 

6 
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In addition to these materials, each school was asked to provide 
crayons and a tape cassette player. 



The field test version of the teacher's manual suggested that at 
least one lesson should be taught each week, the lessons to be supple- 
mented by teacher- led optional enrichment activities at the teacher's 
discretion. 

Tape-led Lessons > ' 

Each of the 24 tape-led lessq^ thesame format. Materials 

are distributed and the teacher turns on the tape, placing the player 
so that students can hear without difficulty. The teacher on the tape 
introduces the lesson by stating the lesson objectives, reminding stu- 
dents that they may color their Storypages, and telling them to listen 



carefully. The dramatized story installment follows. The teacher on the 
tape then comments on the story, instructs the„ students to turn to their 
Gamepages and draw or write according to her directions, and provides 
feedback for their responses. Finally, students turn to their Review- 
pages to which they respond by checking "yes" or "no" to five questions 
presented by the teacher on the tape. Students check their own responses 
according to feedback given by the teacher on the tape. When the tape 
finishes — having taken approximately 30 minutes — the classroom teacher 
collects the materials and may conduct an optional activit}^ as suggested 
in the manual if he or she wishes. 



III. PROGRAM IMPLEMENTATION 
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Students work individually for the tape- led lessons of Unit I. 
In Units II and III students work in small (six-person) groups. Al- 
though the format of all tape-led lessons is the same, in Units II and 
III there is slightly less tape feedback for the activities since -stu- 
dents are directed to discuss their responses within their groups. 
Heart to Heart Talks 

Each of the 6 Heart to Hearc talks follows the same format. The 

classroom teacher explains the purpose of the discussion. 

This is a time to talk, heart to heart, about what 
you've been hearing about on the tapes, -and what 
you've been doing in the Heartsmart games here in 
class. This is a time to talk about how any of that 
reminds you of things in your own life. Talk one 
at a time; listen carefully to the person who is 
talking; speak up as loudly and as clearly as you 
can when you are talking. (Berzon, 1977).- 

Referring to the lesson notes given, the teacher reads or paraphrases r. 
main p,ointy a value judgement based on a concept presented in the story, 
and asks a question ^ usually requiring students to give opinions, illus- 
trative examples, or "real life" descriptions of situations related to 
the main point. Suggestions are given in the teacher's manual for ways 
iii which the teacher can encourage all students to participate. An aver- 
age of five main points, each with a related questions, is presented for 
each Heart to Heart talk. No timelines are suggested for these talks; the 
teacher's manual implies that all given main points and questions should 
be covered. 

In thex^introductory guidelines of the manual, the teacher is advised 

to: 



• repeat and reflect students' responses; 

V • give support, possibly by referring to situations 
encountered in the 'everyday classroom, as told 
from the point of view of the teacher and her^/his 
feelings ; 

• vary the questions to allow for both short and long 
responses ; 

o encourage students to discuss thsir personal experi- 
ences only when they indicate they really want to do 
so ; 

• focus the discussion on the given questions. (Berzon, 
19 77). 

Teacher- led Activities 

Suggestions for teacher-led activities are included in the teacher's 
manual for each of the 24 tape-led lessons. K total of 154 activity 
suggestions are given, all related to Heart^mart concepts, but also 
designed to be integrated with other curricular subjects. The teacher 
is advised to use any or all of these activities at his or her discre- 
tion, combining them with studies in other subject matter areas, assign- 
or* > 

ing them for free time, or using them to illustrate a Heartsmart concept 
brought up by students '/behavior.. ^ 

Small Group Activities ^ . ' 

At the end of Unit I the teacher is instructed to form six-person 
groups for the remainder of the program. Each student is asked to list 
5 students with whom he or she would like to work. The teacher uses 
these lists, and the criteria below to form six-person groups. The 
criteria are: 

• each group should contain a range of ability levels; 
each group should contain at least one independent, 
task-oriented member; 

0 students who tend to be imonopoiizing and disruptive * 
should be placed in groups with students not likely 



9 



ERiC : 



to cooperate with* them in efforts to resist or compete 
with the tape; 

• students who tend to be particularly passive should be 
distributed throughout the groups. (Berzon, 1977). 

The teacher is instructed to appoint a leader for each group. 

Subsequently, every four lessons, students within each group may elect 

their own leaders. A leadership training tape is provided; the teacher 

is instructed to play .this tape to new group leaders before they assume 

the leadership role. The purpose of the tape is to "familiarize the 

leaders with' lesson materials and procedures ... so that during the les- 

son they will not have questions about what they should be doing" 

(Berzon, 1977). 
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CHAPTER THREE: EVALUATION PLAN AND PROCEDURES 

This chapter delineates the major goals and questions of the field 
test, describes' the field test procedures, and discusses limitations of 
the study. 

!• FIELD TEST GOALS 

The major goals of the field test were t^o conduct both formative 
and summative evaluation of the Heartsmart instructional program. With 
respect to formative evaluation, the assumption was that revisions and , 
development work based on previous "tryouts"^ and pilot testing of the 
program would not be undone by the field test; the role of the field 
test would be tha^ of providing information for refining as many of the 
basic attributes of the program as needed to $e effective in "real" 
instructional settings. - ^ ' 

Summatxve evaluation was primaT;-ily conceived 'in terms of assessing 
program effectiveness. "Effectiveness" was interpreted as the extent 
to which the program accomplished its instructional goals and»objec- 
tives. Although formative evaluation constituted an important goal of 
the field test, and considerable effort, resources, and time would 
eventually be consumed in this task, the field te.*:t was designed to em- 
phasize summative (i.e., goal-based) evaluation priority of the 
field test was determined in part by the fact that 1976-77 would be 
the final year in the development cycle of the program and the initial 
stage of product dissemination. The OE/NIE Joint Dissemination Review 
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Panel was considered as a jpossible cornerstone for initiating a possible 
broad dissemination effort, * > , 

II. i FIELD TEST QUESTIONS 

According to the "countenance" model of evaluation proposed by 
Stake (1967), program evaluation involves taking into consideration 
three things: (1) antecedents; (2) transactions; and (3) outcomes. In 
evaluation work,. Stake proposes that each of these be described as 
fully as poiisible, and that within each as well as between .them there 
are important relationships to determine; in Stake's model, these are ' 
referred to as congruence or contingency relationships* A congruency 
relationship involves a comparison between program intentions and 
observations in eac^i of the above three areas. To be fully congruent, 
intended program antecedents, transactions, and outcomes would be 
closely matched with what actually happens. A contingency relationship 
denotes interdependence among these three areas. Educational outcomes, 
for example, frequently come about as a result of given program 
antecedents and transactions. The purpose of the field study test is 
to provide descriptive and evaluative information relevant to selected 
program congruencies and contingencies. Figure 1 shows the general 
model which guided the field test; represented in this model are Stake's 
three program evaluation categories, two -^valuation tasks, and two 
field test/evaluation goals. 
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DESCRIBE . JUDGE ' 

EVALUATION TASKS 



Figure 1: Program Field Test Model 
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Antecedents 

According to Stake, antecedents are those conditions existing 
prior to teaching and learning which relate to program outcomes* Four 
major antecedents are considered and/or described iti this report: (1) 
the field test sites, consisting of students and teachers who partici- 
pated in the field test, and the community environments within which 
the field test was conducted; (2) an orientation session conducted for 
teachers as preparation to teach the course; (3) the instructional ^Xo- 
gram, including /ifts goaiTT organization and content; and (4) the social 
environment of Ithe classroom. \ ^ 

Two ^^^^e principal field test questions and concerns which will' 
be discussed at length in this report relate to^the contents (i.e., 
intrinsic quality) of the program. The Heartsmart Adventures program 
was- designed to enhance interpersonal skills among 9-11, year old 
children* One of the primary means by which the program attempts to 
achieve interpersonal skill objectives is through group interaction. 
Most of this interaction is carefully structured, and takes place 
among members of small groups of students without intervention from the 
teacher. Some of th6 interaction, structured to a certain degree, 
takes place among the members of the class and the teacher. Because 
the objectives of the program are primarily, affective, and possibly 
value--laden, and the means of promoting them involves expression of 
feelings and translation of feelings into actions or behaviors, there 
is a potential possibility that the program could produce harmful 
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educational and/or psychoJ.ogical effects. With this concern in mind 
the field test attempts to address the following major question: 



QUESTION 1: To what extent might the program 

be harmful to learners with respect 
to educational and psychological 
effects? 



This question was addressed by means of an extensive review of 
course materials and the teacher's manual by selected outside experts. 
The review procedures and results of the review are reported in Sectio^ 
I of Chapter Four. 

The second major question of the field test i^ related to social 
-fairness of the materials. Research for Better Schools follows a 
consistent policy witi^ respect to products it develops: instructional 
materials ought to be free from racial/ethnic and sex bias and that 
racial/ethnic minorities should be equitably represented according to 
RBS **af f iriTiative action'* standards. RES' is committed to make appro- 
priate revisions in materials which are judged as falling below estab- 

r. 

lished standards. With" this concern and objective in mind, the field 
test: attempts to answer the following two-part question: 



QUESTION 2: Are racial/ethnic minorities and both 

sexes fairly represented in the program 
materials, and are those materials 
^ unbiased in their treatment of sex and 
racial/ ethnic differences? 



Transactions 

The transactions of a program include all those interactions that 
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occur in the implementation of a program which may affect educational 
results. In terms of the Heartsnart program, transactions which 
were deemed important to describe and evaluate relate to the way the 
teacher administered the program and to classroom social interactions 
involving students and teacher. The latter is especially important in ' 
that the program was designed to bring about increased interpersonal 
effectiveness among pupils through reinforcing certain kinds of 
communication and behavioral interaction. An additional field test 
question emerges from these considerations: ^ 

-A 

- QUESTION 3: Were all essential parts of the program 
implemented as intended? 

These and related aspects of program implementation are dealt with in 
Section III of Chapter Four. There, the program is described in terms 
of how it should have been implemented according to the teacher's 
manual, and how it was actually implemented; then an assessment of 
"congruency" is made between iutended and actual implementation. 
Outcomes 

According to the "countenance" model of evaluation, the category 
of "outcomes" refers to the consequences of educating — immediate and 
long-range, intended and unintended, cognitive, affective, and 
behavioral. This field test report focuses on selected outcomes of the 
program and attempts to relate them as far as possible to specific 
antecedent conditions and transactions. Both congruency and contingency 
evaluation questions are considered. Congruence questions ask whether 

1& 
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the educational outcomes of the program are consistent with intentions. 
Questions 4, 5, and 6 illustrate this type. 



QUESTION 


4: 


Was the prograiif educationally appealing 
and worthwhile to teachers and pupils? 




QUESTION 


5: 


How well did students perform on 
instructional tasks of the program, 
which reflect achievement in understand- 
ing and knowledge acquisition? 




QUESTION 


6? 


How effective was the program in terms 
of accomplishing its major instructional 
goals and objectives? 



In contrast, contingency questions are concerned with determining 
the primary factors which produce or are related to obtained educational 

results. Question 7 below is concerned with program contingencies: 

\ 



QUESTION 7: What factors among antecedent conditions, 
and program transactions relate to the 
nature and extent of program effective-- 
ness? 



1 V 

Underlying these questions is the common concern for describing 

program outcomes and/or evaluating some aspect of program effectiveness, 

'{ . ■ . ' 

The differences among these questions are worth noting, however. The 

notion of "effectiveness" as used in this report is referenced to the 

instrucjtional goals and objectives of the program* As previously 

1 

discussed, the goal of summative evaluation is to provide information 

1 

about program effectiveness, as determined by how well the program has 

i 

accomplished its stated goals and objectives. Questions 6 and 7 deal 
specifically with program effectiveness; results bearing on these 
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questions are reported in Section VI of Chapter Four. Question 4, 
which focuses on program appeal/acceptability, may relate to program 
effectiveness but does not provide direct evidence of student learning; 
therefore, Section IV of Chapter Four gives special attention to 
results related to this question. Question 5 expresses interest in 
measuring student achievement, which implies effectiveness. The major 
reason for treating this question apart from the question of program 
effectiveness is simply that knowledge acquisition was not stipulated 
by the developer as a general goal for the program. Results on 
student achievement are reported in Section V of Chapter Four. 

III. FIELD TEST DESIGN AND SAMPLE 

This section describes the methods and procedures which were 
employed in the field test. Emphasis will be placed on describing the 
methods and procedures used to assess program effectiveness (i.e., 
summative evaluation). This section concludes with a general synthesis 
of the major goals and questions of the field test and the field test 
procedures. 

Summative Evaluation Design 

The design for the evaluation of program effectiveness was 
essentially a quasi-experimental design (Campbell & Stanley, 1963) in 
which groups instructed in the program would be compared with non- 
instructed groupsJ The significant features of this design are the 
use of comparison groups, (i.e., non-instructed pupils), pre- and 
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post test measurements, and ^on-random selection and assignment of 
subjects to instructional and non- instructional groups. It is 
generally acknowledged in the literature on research design that non- 
random assignment of subjects to treatment conditions has important 
^ implications for data analysis and for what types of conclusions can 
be drawn from evaluation results, (These implications are discussed 
more fully in this chapter in Section IV: Limitations.) Because 'two 
of the four primary measures used to assess program effectiveness were 
administered only- as posttests, the summative evaluation design might 
be more accurately described as a modified quasi-experiiaent. 
Sampling Procedures 

A three-stage sampling procedure was necessary in selecting sample 
groups for evaluation study: (1) selection of school districts; 
(2) selection of classes (teachers) within schools; and (3) selec- 
tion of pupils within classes. Control by RBS over the selection 
process was limited at each stage. It was necessary and proper to seek 
consent and cooperation from district and school personnel as well as 
parents in order to implement the evaluation design. 

A primary goal of sampling was to obtain heterogeneous samples 
with respect to residentiar* (i.e. , urban-suburban), racial/ethnic, and 
socioeconomic characteristics* Without sample diversity, the ability 
to generalize the findings of the field test to diverse target 
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populations with similar characteristics would, be. severely limited. 
The sampling procedures attempted to ensure that pupils of varying 
background characteristics would be included in the sample, although 
would not necessarily, simply by inclusion, be representative in a 
statistical sense of the national or even regional population. 
Judgmental (or purposive) rather th^n probability sampling methods 
were employed in order to obtain heterogeneous samples on the above 
dimensions. 

Another sampling consideration (or constraint) was related to the 
geographical proximity of field test sites to RBS. Because the field 
test plan called for on-site observations of experimental classrooms, 
the location of school sites had to permit easy and economical access. 
With these general sampling objectives in mind, the procedures used to 
derive a sample for the field test at each successive stage will be 
described. 

Districts/ Schools « The starting point for school selection was 
to identify school districts in the Philadelphia area which met the 
above, sampling criteria. Five school districts were identified; one 
in Philadelphia, and four in suburban areas surrounding Philadelphia. 
An Assistant Superintendent in charge of curriculum, or a Director of 
Elementary EducsCion was contacted by RBS. The Heartsmart program, RBS 
and the nature and purposes of the field study were described during 
this initial contact. Two of the five suburban school districts were 
not able tc accommodate the field test. The reraaining school districts 
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were sent a follow-up letter and sample program materials for review. 
The Philadelphia School District department of evaluation requested, 
and was given, a description of the proposed field test and copies of 
the measures to be used. In Philadelphia it was necessary to obtain 
consent from both the department of curriculum and the department of 
evaluation. Subsequently > district administrators recommended elemen- 
tary schools to serve as field test sites, .As a result of a series of 
contacts and meetings held between members of the RBS staff, school 
principals, and two fourth grade teachers selected by^the principals 
within each school, commitments were secured from seven schools to field 
test the program. Three Schools were in Philadelphia; three were in a 
district just outside Philadelphia; and one in the urban fringe. 

Classes , During the meetings held at the individual school level, 
the matter of which teacher or class would serve as the experimental 
group and which the comparison had to be decided. The decision rested 

largely with the teachers themselves. The best policy to follow at the 

i 

time was to permit teachers the autonomy to decide which group to be in, 
rather than for RBS to insist on assigning a particular teacher to the 
treatment condition. It was hoped that this procedure would capitalize 
on teacher motivation and interest and would, in effect, create more 
positive conditions for the field study. 

The field study sample consisted of fourteen self-contained class- 
rooms, two from each of seven schools, all of which were pre-*existing 
at the time of introducing the Hearts/nart program. It can be assumed 
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that a socio-cultural climate had already been established in each 
classroom before the pretests were administered. This fact suggested 
taking classroom climate directly into account in a contingency analy- 
sis of program effectiveness. 

Pupils . Frequently pupils in classrooms selected for educational 
"experiments" are -a captive audience; neither the., students nor their 
parents are offered a choice of participation. In this evaluation 
study, a procedure known as "parental consent" was followed so that 
parents of children in selected classrooms/schools would be free to 
choose whether or not to .have "their children take the program. In 
accordance with policies, guidelines, and procedures developed by RBS 
to meet federal government regulations on the Protection of Human 
subjects, RBS designed a parental consent form and prepared materials 
(i.e.. Information Pages) which described the course and its potential 
benefits and risks to children. These materials were sent home to 
parents who were requested to return the consent form to the school. 
A signed form would indicate parental approval to have their children 
take the course. Many of the forms returned contained enthusiastic 
comments about the Heartsmart program; a small number of parents 
expressed, some reservations about their child^s participation, but 
nevertheless, gave their consent. Only one pair of parents did not 
agree to have their child involved in th^ course. 

The parental consent procedure was successful in that it did not 
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result in large' numbers of withdrawals that might either have jeopardized 
the field test or, in a lesser way, have seriously biased the sample. 
(Parental consent materials may be found in Appendix A). 
Sample Description 

In the following discussion, schools, teachers, and pupils parti- 
cipating in the field test are described. 

Districts/ Schools . A total of seven schools, from three districts, 
served as field test sites. Three schools were located in Philadelphia 
suburbs, one in the suburban fringe, and three In inner-city Philadel- 
phia. Attempts were made to obtain specific demographic data (e.g., 
family- median income, educational attainment, racial origin) on the 
local communities in which the field test sites were located. Several 
statistical and non-statistical sources were consulted, including the 
following: Pennsylvania Department of Commerce M-5 Reports; Statistical 
Abstract for Pennsylvania, U.S. Department ^^of Commerce (City-County" 
Data Book), Statistical Abstract of U.S.; U.S. Census Bureau Data of 
1970; and City Planning Commission (Tract Group Profiles). These 
sources provided either out-dated information, or statistical break- 
downs using geographical units. that were much too gross f or^providing 
meaningful comparisons among field test sites. In view of 'the dif f i-- 
culties in obtaining more precise descriptive data on school communities, 
largely impressionistic (i.e., obsefrvational) data must suffice as the 
basis for describing field test sites. A\rough estimate of SES charac- 
teristics was- made by observing things such as size of housing, yard 
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space, type of housing, model and year' of automobiles in the neighbor- 
hood, and general neighborhood condition. 

The three suburban schools are located in predominantly White 
neighborhoods of varying socio-economic status : lower-middle, middle, 
and upper-middle income levels. The school in the suburban fringe is 
predominantly White, 'in a neighborhood of varying socio-economic status. 
Two of the three Philadelphia schools are situated in virtually all- 
Black neighborhoods and appear to be low in socio-economic status. The 
third school is .predominantly Black with a significant concentration of 
White pupils. This school is'^located in integrated Black-White neigh- 
borhoods of varying (i,e,, upper-middle to low) socio-economic status. 

Teachers. Fourteen teachers participated in the field test: seven 
experimental class teachers and seven comparison (non-program) teachers. 
In the experimental classes, all teachers were female; two were Black 
and five were White, Six of the seven comparison classes were taught 
by four White and two Black teachers. Two comparison class teachers 
were male, one Black and one White, In the remaining comparison class, 
a chailige from a Black female to a White female teacher occurred during 
the field test; this shift caused no procedural problems and is assumed, 
to be inconsequential, 

Noa^background data were cpllected for comparison class teachers. 

Experimental class teachers had a combined total of 73 years of 
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^teaching experience, averaging slightly over 10 years each. No 
experimental teacher had previously- taught fewer than three or more 
than 16 years. 

Pupils . A total of 394 pupils were subjects in this field test. 
Of this number, all but 10 wer^fourth graders. It was discovered 
after the field test began that one of the experimental classes 
included 10 f if th .gtaders. In Table 1, a breakdown of the sex and 
racial composition in each school by experimental ys. comparison 
classes is given. 

Class size figures are based on the initially selected subject 
pool (i.e., class rosters provided by teachers) and do not reflect 
sample attrition. Subject loss from original sample pool during 
the field test was minimal (i.e., estimated at 12) and therefore is 
not likely to have a biasing influence on the evaluation results. 
More importantly, no subject dropped from the field test because of 
program-related problems. 

The sample is characterized by balance and diversitjy. Class size 
is nearly equal between experimental vs. Comparison classes within" 
each school. Distribution within sex and race categories is also 
fairly evenly matched between groups within schools. The two possible 
de:piartures from balance were f ound> in school E where the experimental 
class contained roughly 15% more White pupils than the comparison class, 
and in school G where the experimental class included 12% more males 
than females compared with the comparison class. These class 
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Table I 



Description of Student Sample by Sex, Race, and 
Treatment Condition 









SEX 


(%) 


RACE (%) 


School 


Class 


N 


Male 


Female 


White 


Black 


Oriental 


A 


Experimental 


25 


48.0 


52.0 


76.0 


24. '0 


0.0 




CoTnn AT" i Q o n 


22 


SO 0 


50 0 


68.2 


31.8 


0.0 


B 


Exp er imental 


28 


50.0 


50.0 


96.4 


0.0 


3.6 1 




v^omparxson 


9Q 


Aft 7 


J i. / 


100.0 


0.0 


0.0 1 


C 


Experinental 


28 


46.4 


53.6 


96.4 


0.0 


3.6 j 










44 4 


96.3 


0.0 


3.7 


D 


Experimental 


24 


54.2 


45.8 


95.8 


4.2 


0.0 




Comparison 


25 


60.0 


40. 0 


92.0 


8.0 


0.0 


E 


Experimental 


32 


46.9 


53.1 


25.0 


75.0 


0.0 




Comparison 


32 


50.0 


50.0 


9.4 


87.5 


3. 1 


r 


£iXpenmenca± 


7, 1 




ft 

J'+ . o 


0.0 


100.0 


0,0 




Comparison 


31 


51.6 


48.4 


0.0 


1 A A A 
100. 0 


0.0 


G 


Experimental 


29 


44.8 


55.2 


0.0 


100.0 


0.0 . 




Comparison 


31 


32.2 


67.8 


0.0 


100:0° 


0.0 


Total 




394 


48.8 


51.2 


54.0 


45.0 . 


"1.0 


This 


group includes 


e 

ten fifth grade 


pupils . 
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differences, however, are not large in actual nuinbers'of pupils^ The 
balance obtained in class size,. sex, and race^does noty of course, 
imply initial statistical equivalence on measured program or even non- 
measured progTam-r elated variables. 

The. total sample is nearly equally divided between Black and White 
pupils, and male* and female pupils. However, this should no't mask the 
diversity \Jhich is found among the seven schools with respect to race, 
and related characteristics such as the urban/suburban split. Sample 
pupils in schools B, C, and D are over 90% White, whereas sample 
pupils in schools F and G are 100%' Black, Schools A and E have- racially 
mixed sample classes. • 

Although systematic data were not gathered on ethnic background 
characteristics . of the sample, it is worth noting that children from 
Armenian, Po!' ishi, and Spanish heri^.age are also included. These ethnic 



groups were not 

Upon reques; 
special handicap 



concentrated ^in any one school or treatment condition. 
It, teachers provided information on children with 
^ (e.g., hearing, sight, Tangeage) .and serious behavior 
problems which coUld impede their learning in the .program. A small 
number of foreign-born pupils were Identified as having language 
problems, ' but. not serious enough id inval^^ate their test scores on any. 
evaluation measures. Another small number of children'were. judged by 
their teachers as having social behavior problems.* • Jeach^rs felt, 
however, that these children might benefit most from the program i 
There was no educational or psychological reason to delete these pupils 
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from the field test, especially after pa«^ts gave their consent, 

IV. TEACHER ORIENTATION 

As a part of implementing the field test, teachers and school 
principals were invited to RBS for a one day orientation session de- 
signed p^i^marily to ^escribe the program in greater detail, demonstrate 
how to teach the program, and respond to questions and discuss general 
problems related to the field test. Betty Berzon, the program developer, 

was present to conduct the teacher training and to present the philo- 

) ■ ■ ■ 

sophical and theoretical background of the program. The agenda of the 
teacher orientation session can be found in Appendix B. Following the 
orientation, participants, including RBS program staff, completed an 
evaluation form which was designed to assess the effectiveness of the 
teacher orientation, A tabulated summary of these results 'if presented 
in Appendix B. The'ro^ults generally indicate that the teacher orienr 



tation. session was successful. 

V- FIELD TEST MEASURES 

/ 

The evaluation measures and data sources are br/iefly described 
in terms of the following categories: (1) Opinion /Surveys; (2) Obser- 
vation Forms; (3) Interviews; (4) - Procedural Form^ /Guidelines ; ^5) ' 
Criterion-Referenced Tests; and (6) Standardized^ Measures. All data 
collection devices are "included in Appendix C- ' 
Opinion Surveys 

Opinion survey forms were completed by 'teachers arid^'s^tudents. 
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Teacher Opinion Surveys ^ Four Teacher Opinion Surveys (TOS) were 
designed to obtain descriptive and evaluative information from experi- 
mental teachers about (a) individual lessons (TOS Form' //I) , (b) Heart 
to Heart talks (TOS Form //2), (c) individual course units (TOS Form 
#3), and (d) the course as a whole (TOS Form //4) . The content of 
suirvey questions primarily dealt with program implementation, program 
worth, classroom process, and course impact. Most survey questions 
were of the fixed alternative type; a few questions called upon teachers 
to ..give short answers. Host data gathered from these four surveys 
were intended primarily for formative evaluation purposes. 

Student Opinion Surveys . These surveys were developed to ascertain 
the opinions of experimental students regarding Tape-led lessons 
(Survey A), the Heart-to-Heart talks (Survey B) , and tihe Small Groups 
(Survey C) . These surveys were designed to be answered in 1-3 minutes. 

The majority of questions asked students about how much they liked or 

didn't like^^somfething and/or what they liked or didn't like about that. 
These survey data were also ^thered for formative evaluation purposes. 
Observation * forms. ^ 

Selected tape-led lessons and Heart-to-Heart talks were observed 
in each experimental classroom* Observation forms and question sheets- 
were constructed separately for lessons and Heart-to Heart talks, but 
shared a common purpose: to describe and evaluate program implementa- 
tion in the areas of teacher and student behaviors. Construction of 
observation materials involved searching the teacher's manual for 
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directions the teacher was supposed to follow in teaching the course, 
and for descriptions of expected student behavior. Based on this 
search, a series of questions was written. to be answered through 
classroom observation. Observation data were intended to serve 
formative evaluation needs and to provide important information con- 
cerning possible contingencies related to program effectiveness. 
Interviews . 

Interviews were conducted with teachers and students. 

Teacher Interview . At the end of the field test, experimental 
teachers were interviewed about their opinions and observations con- 
cerning varied aspects of the course. The interviews were designed 
for 45-60 minutes. The following is a list of content areas covered 
by the interview: (a) reaction of parents and teachers to the program 
(b) experiences with other instructional programs similar to The 
Heartsmart Adventures^ (c) student reactions to the program, and (d) 

use of Heartsmart concepts and activities with experimental classes 

\ 

and other classes taught. The results of the interview providedX^ 
further information upon which to base program revisions. 

Student Interview . A 10-15 minute interview was conducted with 
three to five pupils selected from one of the ^mall groups in ^ 
each experimental classroom at the end or* the field test. This inter- 
view selection -process was tantamount to being random in effect. The 
interview was' designed to assess the nature and extent of program 
•impact (i.e., effectiveness) on students. The primary areas 
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investigated in the interview related to expression of feelings — what 
feelings were expressed, how often, and in what situations. Interview 
results were intended for suinmative evaluation purposes. 
Procedural Forms/Guidelines , 

- Teachers were provided a Field Record Book which contained a 
Student Attendance Record, Course Schedule, and a Teacher Information 
Sheet, The first two forms were designed to maintain a record of 
program implementation throughout the fi.eld test. A small amount of 
teacher background data was gathered by the Teacher Information Sheet 
for general information and reporting purposes. 

Guidelines were employed to facilitate the evaliiation of intrinsic 
quality (i.e^, harmlessness, social fairness) of the program. Outside 
consultants who performed a review of the program were furnished with 
a set of^ questions/tasks to assist the review. Section I of Chapter 
Four prpvides more detailed information concerning review guidelines. 
Criterion-Referenced Tests . 

Two tests were constructed by RBS staff to assess the extent to 
which content objectives of the program were achieved by the end of the 
field test. The two tests, used as posttest measures for experimental 
afid comparison classes, are: What^s Happening? and What Would You Do? 

What's Happening ? This test conaieti of eight cartoon illustra- 
tions depicting characters (e.g.» friends^ parents^ grandparents) in 
situations presiimably familiar to the experience of fourth grade 
elementary school childrani Each cartoon is followed by a aer.las of 



31 

44 



short-answer, open-ended questions designed to assess student learning 
in four areas: (1) identifying feelings; (2) identifying ways of 
expressing feelings; (3) demonstrating how to find out another person's 
feelings by asking that person; and (4) determining the cause of 
actions or feelings. Like the What Would You Do ? test, this measure 
attempted to be free of program-specific language that could possibly 
bias evaluation results in favor of experimental pupils.. Based on in- 
house reviews, this measure was considered to have "face" validity. ^, 

What Would You Do ? The summative evaluation plan originally 
called for a measure of student achievement (i^e., "mastery test"), to- 
consist of a selected sub-set cf items from "Reviewpages." Each 
Reviewpage contained five items designed to assess information recall 
of program concepts taught by tape-led lessons. A preliminary set of 
items was selected (and modified as needed) from Reviewpages to be 
tested on students who were instructed in the Heartsmart program, 
during the 1976 Pilot Test, and on a comparable group of non-instructed 
students attending the same school. The results of the test revealed 
that most items were too easy, consequently producing ceiling 
effects, and that the power of items to differentiate the two groups 
was negligible. The decision was made to drop the idea of a strictly 
cognitive measure of pupil achievement for purposes of sunnnative 
evaluation. 

A test called What Should Be Done ? was then considered as a 

■ \ 

possible alternative criterion-referenced measure* This test was 
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initially conceptualized and designed for the 1976 Pilot Test by Dr. 
Adrianne Bank from the Center for the Study of Evaluation (University 
of California at Los Angeles). It was necessary, however,. to modify 
the content of test items substantially so that they better reflected 
the instructional goals of the field test version of the program. The 
name of the test was also changed to What Would You Do ? to remove the 
implied dogmatic and moralistic tone of the program. 

The revised test consists of 18 multiple choice items. Each item 
describes a problem-oriented situation involving , young- children and 
other persons such as friends, parents, and siblings. Most of these 
situations were created from "real-life" experiences of children who 
were involved in the 1976 Pilot Test. For each item children are 
asked what they would d£ in these situations. Three response option^ 
representing dif f erent ^inds of actions are given: one option^ was in- 
tended to^ reflect an objective-referenced response; either or both of 
the remaining options (i.e., distractors) either were not related to 
program objectives or were statements of anti-objectives of the 
program (e.g., hiding feelings, guessing instead of asking another 
person's feelings, dependence on authority, physical aggression against 
another person). The course objectives presmably measured by this 
test relate to: (1) expression of feelings (e.g., anger, loneliness, 
fear); (2) translation of feelings into actionsj and (3) expressing 
desires and requesting actions of another to accomplish desires. 

A split-half reliability coefficient of internal consistency was 
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calculated for this test by correlating odd numbered and even-numbered 
items o When the resulting correlation was corrected for test length 
by the Spearman-Brown formula, the obtained reliability coefficient 
was 0.795, The test was also factor analyzed in order to discover 
any underlying program-relevant dimensions which could possibly be ' 
developed into separate indices. Factor analysis produced only one 
psychogically meaningful factor which accounted for 93% of the variance 
in the test items. Based on the above sets of results, it was appro- 
priate, thenj to represent a subject ''s score on the teat by the total 
number of correct .answers. 

The two measures described above. What's Happening ? and What Would 
You Do ?, were specifically constructed to assess selected aspects of 
program effectiveness. No other measures come closer than these two 
to meeting the summative evaluation needs of the field test. It is 
ijriportant to emphasize, however, that t »ese tests do not purport to 
measure pupil behavior in actual interpersonal situations; they only 
presume to measure tendencies (or predispositions) toward behavior at 
least in the types of situations represi?T7t in both tests. The extent 
, of any relationship between these behE- yral tendencies and correspond- 
ing behaviors in real situations is not known. Program effectiveness 
must, therefore, be interpreted as ths extent to which experimental 
students show evidence of being able transfer more of that which is 
Ifearned in the program than compariscrt' students to a paper-and-pencil 
representation of real-life interpersonal events, 
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Standardized Measures . 

Two additional measures .were selected to describe and/or judge 
the program's effectiveness: (1) the Fundamental Interpersonal 
Relations Orientation Behavior Scales (FIRO-BC, Children's version); 
and (2) the My Class Inventory ^ These two measures and tl.e criterion- 
referenced tests described above are primarily distinguished on the 
basis of content specificity. , The latter two tests are more closely 
aligned with the instructional coritent of the program. In contrast, 
the standardized measures, used for pre- and posttesting, are less 
direct in their assessment of program fulfillments and, to some" extent 
are used in the field test ^s vehicles for exploring program-relevant 
differences between experimental and comparison groups, 

FIRO-BC . The FIRO-BC is a self-report, paper-and-pencil measure, 
of how one acts, or is disposed to act, and how one wishes others to 
act in interpersonal situations. It is the children's version of the 
PIRO-B scales used for adults. The same methodology was used to con- 
struct these two measures; they both rely on the same theoretical 
considerations and measure the same dimensions. 

The FIRO-BC, like its parent version, is comprised of six, 'nine- 
item scales derived from three areas (i.e., needs) of interpersonal 
rel-^nicns, laclusion. Control, and Affection, and two dimensions. 
Expressed and Wanted, as shown in Figure 2: 



DIMENSIONS 



INTERPERSONAL 
NEEDS 





EXPRESSED 


WANTED 


INCLUSION 


E -. I 


W - I 


CONTROL 


E - C 


W - C 


AFFECTION 


E - A 


W - A 



Figure 2, FIRO-BC Scales, 

as derived from Interpersonal' 

Needs and Behavioral Dimensions 



The fundamental interpersoaal needs based on the FIRO theory 

(Schuiiz, 1966) are as follows: ^ ^ 

1. .The interpersonal need for Inclusion is. the need to establish and 
maintain a satisfactory relationship with people with respect to 
interaction and association. Some terras that connote various as- 
pects of a relationship that is primarily positive inclusion are 

associate, interact, mingle, communicate, belong, companl^, com- 
rade, attend to, member, togetherness, join, extravert, pay atten- 
tion to, interested, encounter." Negative inclusion is connoted 
by exclude, isolate, outsider, outcast, lonely, detached," with- 
drawn, abandon, ignore." 



2. 



3. 



The interpersonal need for control is the need to establish and 
maintain a satisfactory relationship with people with respect to" 
control and power. Control behavior refers to the decision-making 
process between people. Some terms that connote aspects of primar- 
ily. positive control are, "power, authority, dominance, influence, 
control, ruler, superior, officer, leader." Aspects of negative 
control are connoted by "rebellion, resistance, follower, anarchy, 
submissive, henpecked, milquetoast." 

The interpersonal need for affection is the need to establish and 
maintain a satisfactory relationship with others with respect to 
love and affection- Some terms thaf connote aspects of primarily 
positive affection are "love, like, emotionally close, personal, 
intimate, friend, sweetheart." Aspects of negative affection are 
connoted by "hate^ool, dislike, emotionally distant, rejecting." 

Two aspects of ^ehavior are assessed by the measure: the behavior 
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an individual expresses toward others and tha behavior an individual 
wants others to express toward him/her - hence, six scales: Expressed 
Inclusion, Wanted Inclusion, Expressed Control, Wanted Control, 
Expressed Affection, and Wanted Affection. 

The FIRO-BC scales were developed using the Guttman technique for 
cumulative scale analyses. Reproducibility coefficients rather than 
split-half reliability coefficients were therefore used as estimates of 
reliability. Reproducibility measures the extent to which a respond- 
ent's scale score is a predictor of one's response pattern. A high re- 
producibility coefficient for a scale indicates the scale is cumulative 
and presumably unidimensional. The usual criterion for reproducibility 
is. .90, meaning that 90% of all responses to a given scale are repro- 
ducible from knowledge of scale scores. Table 2 shows a comparison 
between reproducibility estimates as reported by the FIRO-BC Technical 
Manual (unpublished), and those found on pre-tests and post-tests in 
the field test sample. Alpha coefficients of internal consistency for 
each FIRO-BC scale are also given. 

These data indicate that reproducibility scores based on the field 
test sample are consistently lower than those reported by the test 
developer, and fall below the criterion of. .90. The alpha coefficients 
tend to indicate that the FIRO-BC scales may not measure unidimensional 
tracts in this field test, sample. It is possible to improve repro- 
ducibility, and as a result, the alpha coefficient, by re-scoring the 
test. The decision was made, however, to maintain test integrity on 
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Table 2 



Comparison Between FIRO-BC Test Manual and Field Test 
Sample Reliabilities on FIRO-BC Scales 





:> 


REPRODUCIBILITY COEFFICIENTS 


COEFFICIENT ALPHA 


FIRO-BC SCALE 


Technical 


. Field 


Test 


Field 


Test 


Manual 


Pre-TeSt 
(N = 383) . 


Post-Test 
(N = 383) 


Pre-Tes t 
(N = 378) 


Pos t-Test 
(N = 382) 


Expressed Inclusion 


88 


78 


79 


.63 


.66 


Wanted Inclvisicn 


88 


75 


77 


.66 


.72 


\Expressed Control 


90 


Q 83 


86 


.83 


.84 


Wanted Control 


89 


81 


83 


.76 


.77 


\ 

Expressed Affection 


88 


76 


77 


.61 


.63 


Wanted Affection 


88 


77 


80 


.63 


.67 



3d 

51 

o 

ERIC 



the assumption that any generally reliable differences between experi- 
mental and comparison groups could still be detected by these scales. 

My Class inventory (MCI ), This measure is the elementary school 
version of the Learning ^Environment Inventory (LEI) designed for 
secondary school use. The MCI is an "instrmiient designed to measure 
the social climate of learning of a class as perceived by the pupils 
within it (Anderson, 1973. p. 4)," The scales which comprise this 
instrument were developed from theoretical and applied social-psycho- 
logical and educational researcTi,. Test construction involved both 
expert judgment in classifying items ivito construct categories and 
empirical testing. 

The MCI contains 45 items distributed over five scales: Satis- 
faction (S), Friction (F), Cohesiveness (C) , Competitiveness (CM), and 
Difficulty (D). The first three scales, S, F, and C, were selected to 
be used as measures of program outcomes, referenced indirectly to 
program objectives. The S scale appears to be a measure of generalized 
satisfaction with, the class as a whole. The F scale was designed to 
measure the extent to which a classroom group is characterized by lack 
of mutual respect, bickering among class members, inter-group tension, 
and lack of group cohesion. The C scale is nearly the opposite of the 
F scale; a classroom scoring high on the C scale is made up of pupils 
who know and like each other, and who help one another. ' 

According to its developer, the MCI, is still undergoi^ng develop- 
ment. The reported reliabilities on the S, F, and C scales of the MCI 
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range from .54 to .77. Alpha coefficients calculated on the field test 
sample are compared with those reported in the test manual in Table 3 
below: , 



- Table 3 

Comparison between MCI test manual, 
and field test sample reliabilities 
on three MCI scales 



\ ■ 







My Class Inventory 
Scale 


^ ALPHA COEFFICIENT 


c 

Technical 
Manual 
(N=655) 


Field Test 


Pre-Test Post-Test 
(N=386) (N=383) 


Satisfaction 

Friction 

Cohesion 


.77 
.70 
.54' 


,69 .76 
.62 • .66 
.57' .56 



- These results show that reliabilities of these scales are lower 
in the field test sample on the S and F scales and slightly higher on 
the C scale. Appropriate caution should be taken in judging evaluation 
results using these scales due to relatively low scale reliabilities. 

VI. DATA COLLECTION AND ANALYSIS 

This section describes methods of collecting data, and describes 
and discusses data analysis with distinction being made between 
summative and formative evaluation. 
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Table 4 contains information about (1) timing of data collection; 
(2) agent(s) responsible for collecting data; and (3) data, collection 
procedures,. Most of this information is self-explanatory. There is 
need, however, to describe, classroom observation'methods in more 'detail. 

Observation > A total of 48 observations was made of Heartsmart 
program lessons: 36 of tape- led lessons and 12 of Heart to Heart talks. 
On the average, each school w^; visited for observation purposes nearly 
7 times throughout the field test (i.e., 13-18 weeks). Although four ' 
RBS staff members visited at least two schools each, most observation^ 
data were collected by one individual, since it was thought that students 
and teachers would be less disturbed if they became accustomed to one 
person. Observation procedures were relatively simple: the^ observer 
would typically sit in the back of the room, and listen, watch, and 
record selected verbal and non-verbal events occurring in the classroom. ' 
Observation cu'3s x^er^ provided by a series of directed questions. After 
taking extensive notes of classroom processes, the obs^'yer would pre- 
pare a written respoit^e to these questions based on the observation. 
Observers' classroom notes and answers to questions would then be 
collected and filed for subsequent us^e. 
Data Analysis 

It is here tiHat the distinction between formative and summative 
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i Tabled 

1 

1 

Description of the Field Test Data Collection Method 





Instrument 
Category 


■ Instrument (s) . 


Test. Administration 
Before During After 


Test 
Administrator 


Data Collection Proceduie 


\JUllll\JU JUiVvYO 


TPflfhpT Driinlnri ^mi^vpvc 


A 




Self-Adiainistered bv teacher 




Student^ Opinion Surveys 


X 

I. 


Teacher 


1 For eacli class period five ' 
different subjects selected 
from teacher's class list 


t 

Observation Forms 


■ ^ 

Legion Observation 




RBS Staff 


Classroom observation 


• / 


Heart-to-Heart lalk 




RBS Staff 


Classroom observation 


f 

Interview 


Teacher Questionnaire ■ 


X 


RBS 'Staff > 


Face-to-Face interview 


Schedules 

>' 


Student Questionnaire 


, X 


'rBS staff 


Face-to-Face interview""\ , 


Procedural Forms/ 
Guidelines 

> 


Teacher Data 
Student Record Sheet. 


X 

■ ' X 




Self-Administered by teacher 
Self-Administered by teacher 




Course Schedule 


•■ X ' 




■ Self -Administered by teacher 


Criterion- ^ 
Referenced Tests 


t 

' Whaf's Happening? 
What Would You Do?. 


X ' 
X 


RBS Staff 
RBS Staff 


Test items read aloud to pupils 
/ Test items read 'aloud to pupils 


Standardized^ 
Measures 


FIRO-BC 

My "-Class Inventory , 


, X \ ■ . X 
X ' 'X 


RBS Staff 
RBS Staff 


Test items read ploud to pupils 
Test items read aloud^ to pupils 

I. , 



1^ 



Administered to both Experimental and Goniparison Group subjects. 
Time- ijiterval between pre-post testing was approximately 13 to 18 weeks, depending on the field test site. 

^Except^ons: All subjects 'completed last Heart-to-Heart and group opinion survey forms: 
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evaluation is convenient and necessary to draw. Formative evaluation 
is priinarily concerned with utilizing data for making program 
revisions. Summative evaluation is almost exclusively concerned 
in this report with: (1) utilizing data to describe and judge the 
outcomes of the Heartsmart program in relation to program goals and 
objectives; and (2) utilizing data to describe and judge contingencies 
as they may relate to (1). In this report formative evaluation in- 
volves experimental groups only, whereas summative evaluation involves 
making comparisons between experimental and comparison groups • Data 
analysis procedures which were employed to serve both formative and 
summative evaluation goals of the field test were therefore quite 
different and need to be discussed separately. 

' Formative Evaluation . Data analyses for formative evaluation 
purposes were 'performed by both logical and empirical procedures. 
Content analysis was the principal logical procedure and was used as 
the basis for analyzing intrinsic quality^ social fairness, 

educational/psychological harmlessness) ofXthe program materials. The 
results from the Consultant Review and 'Af f innative Action, reported 
separately in Chapter Four, were based on conterrK analysis procedures. 
Consultant Review was guided by less formal, primarilxy judgmental 
procedures, whereas Affirmative Action followed a more^^rmal set of 
guidelines and rules. 

Descriptive statistics were used in the analyses of student 
achievement, program appeal/accejptability , and program implementation 
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data. Varied analyses of the data provided most of the necessary 
information to determine program revisions, 

Summative Evaluation . Analyses of summative evaluation data were 
performed principally by multivariate analyses of variance (MANOVA) 
and multivariate analyses of covariance (MANCOVA) • In the case of 
pre- and post-test measures, a repeated measures MANOVA/MANCOVA was 
used. This type of statistical approach to summative evaluation has 
important advantages over other procedures (e.g., univariate gain- 
score analysis, analysis of variance/covariance) . First, it performs 
the usual F-test for significant differences between groups on more 
than one dependent variable simultaneously. In the, case of multiple 
dei andent measures or indices on the same set of subjects, one is 
likely to find significant intercorrelations. In an "experimental" 
situation, finding a reliable difference on one dependent variable 
increases the chances of finding a similar difference on related 
variables. MANOVA, however, effectively partials out dependent 
variable intercorrelations and provides a more accurate assessment of 
group differences. In Appendix D scale intercorrelations among the 
FIRO-BC test and the MCI, and the correlation between the WhatVs 
Happening ? and What Would You Do ? tests are given. The large number of 
highly significant correlations amply justifies the use of MANOVA, A 
further, related advantage of MANOVA, in combination with univariate 
statistical analyses, is that it maintains a consistent, moderately 
conservative experiment-wise error rate regaidless of the magnitude of 
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dependent variable correlations (Hummel and Sligo, 1971) • The Alpha 
for determining the statistical significance of multivariate results 
was set at the .10 level; the relatively small group sample sizes 
involved in the statistical comparisons between experimental and 
comparison classes vithin each school permit a relatively non-conser- 
vative Type I error rate. The Alpha level for univariate results was 
set at .05 to take into consideration inflation of the error rate 
per comparison in performing multiple statistical tests. 

Multivariate statistical results presented in this report were 
provided by the Statistical Analysis System fBarr et^ al . , 1976) and 
Multivariance Version VI (Finn, 1977) computer program packages. 

VII. LIMITATIONS/DELIMITATIONS 

This section is motivated by the need to place the findings of 
this field test in a proper cautionary perspective. The purpose of 
this section is to provide a faithful accounting of limitations/delimi- 
tations of the field test study. Limiting/delimiting factors are conr 
,sidered in each of the following areas: field test goals, program 
goals, suramative evaluation design, and instrumentation. 
Field Test Goals 

The major goals of the field test were \to conduct both formative 
and summative evaluation. The question is whether field test results 
would be equally applicable to the final program version which includes 
data based-revisions. The credibility of summative evaluation must. 
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therefore, be dependent, in part, on the nature and extent of these 
revisions. Actual program revisions, as will be documented in Chapter 
Six of this report, were extensive and necessary, but do not greatly 
alter the basic structure, organization, content, or implementation 
strategy conceptualized for the original design of the program. It is 
therefore assumed that if this field test were replicated on the final 
revised version of the program, it is probable that similar findings 
would be obtained. 
Goals of the Program 

A significant and continuous chaLlenge to RBS evaluation staff 
was to be able to identify and comprehend the goals stated for the / 
Heartsmart program. The problem of identifying the program goals was j 
evident in the lack of complete correspondence between the content of ' 
the instructional materials and the goals for the program. The overall 
program goal is to enable students to conduct their interpersonal rela- 
tionships in a more open, aware, and effective manner; this goal may be 
perceived as open to a variety of interpretations. As the field test 
progressed, it became evident that the program goals and objectives 
could not be adequately understood apart from the FIRO theory (Schutz, 
1966). The teacher's manual states that the program is designed to 
teach-^students , among other things, to ''be able to express (by wcrds or 
actions) what they want or don't want (from other children and/or adults) 
in the areas of affection, being included in activities, or being in 
charge (Berzon, 1977, pp. 2-3).'* Again taken from the teacher's manual, 
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"goals of Units II and III involve legitimizing 'acceptance of, and en- 
abling expression of Inclusion..., Control..., and Affection" (Berzoh, 
1977, p. 85). The logical choice of an assessment device to measure 
program effectiveness was the FIRO-BC scales which were designed to 
measure Inclusion, Control, and Affection. However,, making specific 
_ predictions from the FIRO ^heor)^ or from the content of the program, 
turned out to be a complex task. This difficulty was not resolved in 
consultation with the program developer and the originator of the FIRO- 
theory, both of whom expressed uncertainty about how the FIRO scales 
would react to the program treatment. Both consultants suggested in- 
dependently that the best use of the FIRO scales was as an exploratory 
device rather than as a measure of program effectiveness . Why \^as it 
so difficult to generate a set of testable evaluation hypotheses using , 
the FIRO scales when the rationale and content of the program x^. Ljsed 
on the FIRO theory? The actual relationship between the FIRO theory 
and the instructional content for the program is not as strong or direct 
as program staff initially thought. The^rogram does not attempt to 
teach students explicitly to express or want more or less "inclusion," 
"control," or "affection," but does provide au opportunity for students 
to experience these facets of interpersonal phenomena, particularly in 
the sm^ill groups. The influence of the FIRO theory on the Heartsmart 
program is more evident in terms of its organization rather than in 
terms of its instructional content » It would appear that the FIRO 
scales do not provide a strong basis for summative evaluation, contrary 
to RBS original understanding. 
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The point that is hopefully raised by. this discussion is that the 
summative evaluation was approached with certain understandings of the 
program goals and the program's underlying theory. These understandings 
influenced the development and selection of evaluation measures, and 
presumably the interpretation of the results obtained in this study. . 
Had another party conducted the summative evaluation, it is conceivable 
that its understanding of the program and its goals would differ from 
this one, as would i^s selection of appropriate evaluation measures , 
and perhaps, even its evaluation findings. 
Summative Evaluation Design 

In this field test, the assessment of program effectiveness was 
determined by the use . of a quasi-experimental design. The chances of 
incorrectly attributing expected differences in performance between 
experimental and comparison classes to the effects of a program treat- 
ment are greater for this type of design than for true experimental de- 
signs. According to Campbell and Stanley (1963) ,. there are two possible 
competing explanations: regression and the interaction between selec- 
tion and maturation* Since samples were not selected on tY\e basis of 
extreme scores on some measure, and since the sample selection proce- 
dures minimized student self-selection as a factor, regression and 
selection and maturation interaction are probably not very plausible 
alternative explanations. A more realistic concern relates tp the 
effects of possible selection differences on the two criterion-referenced 
tests. It will be recalled that these tests were administered\ only as 
post-tests. If experimental and comparison classes systematically dif- 




fered initially in these variables or related variables measured by 
these two tests, then any posttest differences could be attributed to 
those initial differences • Results from the criterion-referenced 
measures, whether they significantly favor or do not favor the experi- 
mental classes, must be interpreted in light of possible selection 

/ 

u 

differences. 
Instrumentation 

Reliability data were presented for the I^at Would You Do? test, 
the My Class Inventory (MCI), and the FIRO-BC scales in Section V. 
Only the reliability coefficients obtained on the MCI scales were found 
to be undesirably low and, therefore, appropriate caution siiould be 
taken in interpreting results on these scales • 

Earlier the relevance of the FIRO-BC scales as measures of program 
effectiveness was questioned. The FIRO scales in general were construct- 
ed mainly co do basic research on interpersonal phenomena (e.g, , predict/ 
explain social interaction under varying conditions) and to provide 
clinical information to individuals concerning their own interpersonal 
needs. These scales are frequently used in clinical-therapeutic set- 
tings to assist groups or individuals to learn more about themselves and 
their interpersonal behavior. (Occasionally the scales have been used 
to assess outcomes of intensive (e*g. > encounter) group experiences.) 
The FIRO scales were used in this field test for a different purpose, 
which was as an evaluation device for detecting any meaningful experi- 
mental vs. comparison group differences as e function of an educational 
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intervention. No attempt is made to analyze or interpret evaluation 
results from the FiP.O-BC scales in terms of individual scores; all data 
are aggregated across individuals and it would appear that a large degree 
of psychological mear^i?igf ulness of the test results is lost as a result. 
Furthermore, an educational program, such as The Heart smart Adventures^ 
probably should not be expected to bring about deeper psycho-social 
changes (i.e., changes in interpersonal orientation) in children that 
might be affected by longer term and more intensive professional psycho- 
logical services. The FIRO-BC scales may be inherently more sensitive 
to these latter type experiences. 

Finally, the FIRO-BC scales for children have not been as widely 
used as the FIRO-B scales for adults. This field test is the only 
known attempt to use the FIRO-BC to measure the effectiveness of a 
school-directed instructional interventiq^n. Its validity is therefore 
not based on much previous related research experience. 
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CHAPTER FOUR: RESULTS 

This chapter consists of six sections: Consultant Review; Affir- 
mative Action Report;^ Program Implementation; Program Appeal and Ac- 
ceptability; Student Achievement; and Program Effectiveness, 

I. CpNSULTAriT REVIEW 

The procedures and results of consultant review are reported in 
this section. 
Tasks and Procedures 

In accordance with the requirements of the field test design, and 
with a desire to obtain external review of the program, it was deter- \ 
mined that three to four consultants would be invited to review and 
evaluate the program. It was further determined that each consultant 
should be asked either to respond to specif ic questions , or to evalu- 
ate the program from a particular viewpoint so that no one consultant 
should be overburdened by too tiuie-consuming a task. 

The following questions were posed: ,/ 

1. Is this program suitable for students aged nine to eleven 
years old? Is it equally suitable for all students in this 
age group regardless- of race, ethnic background or socio-'* 
economic status? Would students benefit from the program, 
or would they be harmed? 

2. Would the program be an acceptable part of an elementary 
school curriculum? 

3, Could the average classroom teacher carry out the program 
objectives given the materials provided? 

4, The FIRO. (Fundamental Interpersonal Relationships Orientation) 
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theury relates to the individual needs of inclusion, control, 
and affection^ (Schutz, 1966). Is that relationship, and th 
program's interpretation of the theory valid and appropriate? 

Since Dr. William C. Schutz originated the FIRO theory, he was 
invited to respond to question four above. 

Ms. Patricia Ramsdell, Director of Elementary Education for the, 
Upper Darby School District, was asked to respond to the first three 
questions. 

Dr. Yvonne Agazarin, a clinical psychologist, was asked to re- 
spond to questions one and three. ^' 

The consultants were approached in February 1977. All three 
consultants agreed to review the program. Each received a complete 
set of tapes andjj^a copy of the teacher^s manual. Review questions 
were posed, hut no formal forms were used. All reports were re- 
ceived by the end of July, and'^are on file at Researc. for Better 
Schools. ^ 
Report from Dr. William Schutz 

The report begins, *'My overall impression is one of overwhelming 
enthusiasm the program ... is creative and meticulous, theoretical 
ly sound," and later states, "I would anticipate great success, es- 
pecially with an aware teacher." 

' ■ ■ ' f^,_ \, 

Dr. Schutz makes three specific suggestions: 

• Condense the lengthy lists of objectives- 

• Revise the story for Lesson 19 so that it is oriented to power 
competition, authority and influence, i.e., control , rather 
than to attention which is an aspect of inclusion . 
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• Carry through or continue the series of lessons by teaching 
students to take responsibility for their own choices. 



Dr. Schutz concludes with praise for the program and states that 
the presentation of the FIRO dimensions "is both accurate and. enter- 
taining," . . » . 
Report from Ms. Patricia Ramsdell 

In her report, Ms, Ramsdell states that she listened to all the 
tapes, read the student materials and teacher's manual, observed two ' 
lessons conducted during the field test, and, interviewed some teachers 
and students involved in the field test, 

Ms. Ramsdell states, "This is a great program and for that reason 
my suggestions are very minor." Her suggestions are quoted verbatim 
below. 

1. Eliminate the part in which candy is given as 
it seems .to suggest that it is alright to take 
candy frbm a stranger. Tape 1 and ,10. 

2* Eliminate suggestions of running away from home 
and running away to find self or^ recommend that 
the teacher discuss this as make believe and 
not an acceptable real life practice. Tapes 1 
and 10. 

3, On the tape that refers to the fact that it is 
^,k, to do things that are different add — it 
is o.k. ±1^ it doesn't harm other people. Tape 

15. ^ / . ■ 

4, Tone down the child telling parents hov; he feels. 
^ In some of our homes a child would be punished 

for using that tone of voice. Tape 11. 

5, Change suggested grade level to three /or four 
to be determined by the. sophistication of the 
students,' / 

a 6, In the beginning of the tape vary the approach — 

e,g, "Good morning," "Here we go again," "Hello 
again," "Hi Everybody!" I got so tired of the 
same opening phrase! All tapes. 
7. Drop the term "speakeasy." It reminds me of 

prohibition stories — the knocking /on the door 
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, ■ intensifies this feeling. Tape 18. 

8. In giving directions drop some of the "you 
shoulds". All tapes. ' ^ 

9. Change the , review quiz to true and false rather 
than yes or no or remove quiz altogether. All 
tapes, 

10. Design a follow through .guide for teachers. 

ir. Add a lesson on responsibility to others — e.g., 

sharing, carrying, caring, etc. - 
12. Design parent lessons so all -of the family learns 

to express feelings. 

Report from Dr. Yvonne Agazarin 

Dr. Agazarin listened to all the tapes and read the teacher's ' 

■J 

manual. 

She states: 

In my opinion the progra^Ti is generally and specifically 
. valid in meeting the overall goals stated and defined In the 
. introduction* of the^ teacher's manual, • 

I: found the program to be age appropriate as determined 
by the Fry readability scale . , • ' in my opinion, the impact :of 
the program would have a strong, positive influence on the 
, psycho-social growth of the^ children who take it. 

■ ■ . ■ i 

X want to emphasize that my overall evaluation of this' ' 
program is positive •.. Because inner-city children are most' 
subject to social deprivation and sensory deprivation and the 
concomitant impairment of the self-mastery that comes from' 
this, I would consider it particularly important that this/ 
program be given to "inner-city school children. ;/ 

Dr. Agazarin makes many highly specific comments and suggestions, 
some of which are organizational or editorial. For the sake of brevi- 
ty, only her major suggestions are suraAarized hera. 

Teacher's Manual . In general tl^ere is too much repetition, and 
the orgf-nization, for reference purposes, is poor. The style is iri;;on 
sis tent; all copy should be written directly to the teacher. 
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Insufficient trslning and/or direction is provided for the teacher 
on how to. conduct Heart- to-Heart talks. 

A warning to the teacher that students may experience difficulty 
with a given tape-led activity is inadequate. The nature of the dif- 
fdculty should be identified and resolution strategies suggested. \ 

Tape-led Lessons . Lesson 1: It is not accurate to teach children 
that a fast heartbeat means you're scared. Rewrite truthfully. 

Lesson 5: Gamepages and tape-led activity are particularly good. 

Eesson 7: The student competitors in the tapn_-led game do not 
need to be rev/arded by applause. 

Lesson 8: The OOPSER, a computerized robot, should not e::perience 
a nervous breakdown, neither should, it experience feelings. "Children 
should not be encouraged to anthropomorphize machines." 

Lesson 18: "Speak Easy is an unfortunate vocabulary phrase > and 
may even be an unfortunate concept." 

Lesson 19: The characters known as the Hurts run away, and essen- 
tially practice blackmail in order to receive attention. "I'm uncom- 
fortable with rewarding the blackmail .. * the point needs to be 
achieved in some different way." 

Lesson 21: .^In this Heart-to-Heart talk the following point should 

be included: . ' 

It is important to tell people what you want,. .and to risk not 
getting it, rather than not tell, and make sure of not getting 
it. 

y 

Lesson 28: It would be preferable for the character Marsha Me^llow 
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to learn self-re3pect and self-estr -ti, rather than to rely , on other 
people giving her ''kisse..- and hugs'' when she*s feeling upset. . 

• Lesson 30: "I like very mu^h the * hello-goodbye* teaching in 



this lesson.'** 



Groups . ''Random formation of groups would be less trouble for 

/ 

tne teacher, and less risky in terms of idiosyncratic^ reactions to 
sociometric choice." If the method for forming groups is not changed, 
an exercise other than the "spaceship captain" should be used so that 
both task group ;and group maintenance are suggested, and the exercise 
topic relate*^ W^^e closely to Heartsmart lessons. 

More explicit guidance is/ required for group leadership training 
in order to avoid "the imitation of benevolent autocratic behavior." 
Students should be allowed more oppiDrtunity to be group leaders. 

The small groups could bp structured functionally with rotating 
rolas in the areata of inclusion, control^, and affection. In a six-, 
person group three pairs of stindents would take behavior roles re- 



lating to each area. Generally^ sneaking, the inclusion roles' would re- 
late to maintenance through non^toring aspects of cooperation and com- 
parative "communication input." Vrhe control roles would relate to 
task and admir.istration. The aff^ctlou roles would >-elate to maii- 



cenance being "supportive of the group members.": ^ 

Feelings ; Six pair3 of "feelings'* characters are ,:>resented in the 
story: sad, mad, glad, hurt, lonely, \ scared. Add "love." Optional 
activities should be included relating\ to shyness, selfishness, and 
showing-off. 
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'General , Include a set of information pages which may be dupli- 
cated and given to parents or* non-program teachers as and when neces- 
sary. 

Revise the directions to the substitute teacher to allow for more 
input and involvement of the students, 

(See the chapter titled Summary and Conclusions for further dis- 
cussion,) 

II. AFFIRMATIVE ACTION REPORT 

This report describes procedures, considerations and data related 
to portrayal of characters in The Heartsmart Adventures (field test 
version^ spring 1977), This analysis of characterization, in Lerms of 
sex, race or ethnicity, positive-negative depiction, and superordinate- 
subordinate social relationships was conducted-in. accord::nce with the 
RBS guidelines for affirmative action to determine what revisions (if 
any) should be made in order to achieve social fairness in portrayal 
of characters. 
Task 

The document. Guidelines for Affirmative Action in RBS Products 

and Services (RBS, 1976) states: 

Data wlll.be gathered on both narrative and graphic 
aspects of a product wherever individuals are iden- 
tified in a product. The characterization and 
group membership will be recorded according to the 
following categories: 
i, by sex 
ii. .by race/ethnicity 
iii, by disability/handicap. 
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Percent of each group to be represented are suggested as follows: 

i. by sex: 50% male, 50% female - 10% . 

ii. by race/ethnicity: Black 18% t 5% . 

yjfhite 75% r 10% 

Oriental 2% :j: 1% 

HispanD.c-American 2% :p 1% 

Native-American 2% - 1% 
Other 1% 

Balance in characterization is to be determined by considering the 

following: 

i. by positive-negative depiction 

ii. by superordinate-subordinate social relation- 
ship 

iii. by work, play or other activity engagements 
iv. by abilities, interests, or other personal 

traitsi 

Once data has been collected and analyzed, it is suggested that... 

...where the comparison indicates that certain groups 
were underutilized, the product units should be re- 
vised to comply with the standards for group repre- 
sentation. 

Procedures 

--^ ■ 

An affirmative action. plan for The Heartswart Adventures 
program, based on the RBS Guidelines^ was written in July 1976. 

During the spring of 1977, the first stage of the plan was carried 
out: data relevant to affimative action were tabulated, analyzed, 
and reported on. 

An independent evaluator employed on a dif f erent ^RBS project re- 
viewed the report resulting from stage 1, examined samples of the pro- 
gram materials, and wrote a brief report. 

Minority members of RBS were asked to review the, program and 
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attend a meeting to discuss their impressions of program concepts and 
materials. " 

All findings were reviewed by the project coordinator for accuracy 
and/or appropriateness in order to determine what revisions could and 
should be made. 

Only a certain level of objectivity is possible in individual 
evaluation of social fairness characterization. In' order 
to reduce the possibility of biased subjective judgments, the first 
three stages of the analytical review process as 'described above were 
independent of the others, and both white and non-white persons'were 

involve d. 

Limitations and Considerations 

Analysis of group membership of Heartsmart characters is less ex- 
tensive than suggested by the RBS Guidelines because no physical dis- 
abilities are portrayed and ethnic identity cannot be specified beyond 
skin color. 

The dark skin and black hair that are given to some of the people 
illustrated are undoubtedly meant to suggest Afro-American, Asian, His- 
panic, and/or Native American background, but it is impossible to deter- 
mine the developer's "^ntentions. Therefore, the analysis by category 
relates only to sex and white/non-white characterization (the term 
"non-white" being used uo indicate all dark-skinned portrayals). 

In the tables that follow, group membership is tabulated for main 
characters of the story, supporting characters of the story, and 
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characters portrayed in Heartsmart activities. 

Main character > A main character is defined as one who is illus- 
trated in at least twc story pages, and/or who has a speaking part 
totaling a tninimum of twenty written lines. 

Supporting character . A supporting character is defined as one 
who is illustrated in only one story page and/or who has a speaking 
part totaling less than twenty written lines. 

Non-human characters . There is a total of twenty non-human 
characters portrayed in Heartsmart, of whom nineteen have speaking 
parts. Four of the non-human characters are animals; sixteen are 
fantasy creatures. Since all non-human characters have human charac- 
teristics, and are portrayed by humans in the draiaatized tapes, they 
are included in this analysis. 

Activities characters . Characters in the activities portions 
are portrayed on tape and/or by way of illustrations on the gamepv.7e5. 
Only the children, who comprise over 99% of the r.otal number of '.i.-rioe 
portrayals, are tabulated here. (The Feelings, Bessie Bluechip, :U>^ 
the Altogether Alligator have been considered in other tabulations 
relating to main nor-huiaan characters.) Two white hands^ and repre- 
sentational human figures v>hich arr ^-ip.ntified neither by sex nor by 
ethnic backgroir. d , were considered to be of insufficient impottance 
for considerajicn in this study. 

Sex . For several of the ron-human characters, only the voice 
provides a clue for thii identii: '.cation of «ex. In some instances, the 
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voice could be considered either male or female; therefore, the cate- 
gory "indeterminate sex" ^is included. 

Ethnic identity . Illustrations provide the only means for estab-- 
lishing the ethnic identity of human characters. Because illustra': Ions 
are not always provided for supporting characters , it was necesss^'y to 
add the category "indeterminate." 
Representation of Characters by Sex 

Of the 202 characters in the program, 56*4% are male, ^3 1% are 
female, and .5% are indeterminate (See Table 5 ) . These figures sugg.-r^st 
fulfillment of RBS Guidelines calling for 50% luala, 50% fenial- 2. '''^°<" 
However, the figures alone do not adequately describe the balance - or 
imbalance - betwe^^n the sexes. It is necessary to consider the excetx 
of participation of the characters, and to separate human froiu nor^-hu-- 
man characters. 

Characters in the activities are least important in a-; much as 
each character ^.ppears only once, has litl:le depth of character, and is 
rarely Identified by name. Of the 92 activities characters, 65.2% are 
male and 34.8% are female » indicating a small Imbalance Li) favor of 
males. 

Four of the supporting ch^:racters in the story are noc^ human; 
eighty arc human. Of the non- human supporting characters, 3 (75%) are 
male, and 1 (25%) is of radoterminate sex. There are 38 (47.5%) male 
human supporting characters, and 42 (52.5%) female human supporting 

a 

characters, indicating an acceptable balance by sex. 



Table 5 

..Breakdown of all .Characters by Sex and Ethnicity 





Total 
No. 




Sex 




Ethnicity 






Female. 




Non-White 


White 


?* 


Main Characters, 
Human 


9 


6 

6 6.7% 


3 

33.3% 




1 

11.1% 


8 

88.9% 




Main Characters, 
Non-Human 


17 


7'" 
41.2% 


10 

58.8% 






- 


17 
100%' 


Supporting Characters, 
Human 


80 


38 

47.5% 


42 

52.5% 




19 

23.8% 


42 

52.5% 


19 

23.8% 


Supporting Characters, 
Non-Human 


4 


3 

'75% 




1 

25% 






4 

100% 


Activities i^haracters. 
Human 


92 


60 

65,2% 


32 

34.8% 




17 

18.5 % 


30 

3 2.6% 


45 

48.9% 


Total 


202 


114 


87 


1 


37 


80 


85 


Total (%) 


100 


56.4% 


43.1% 
5 — ' 


.5 % 


18.3% 


39.6% 


42% 



*? = indeterminate 
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The main characters in the story are most important. Excluding 
the teacher and the narrator, who have no depth of characterization, 
there are 26 main characters, of whom 9 are human. Six (66.7%) of the 
human main characters are male; 3 (33.3%) are female, indicating an • 
imbalance in favor of males. Main non-human characters include 7 
(41.2%) male and 10 (58.8%) female, indicating an imbalance in favor 
of females. 

It is necessary to consider representation of each group of 
characters in terms of their roles and character traits. Such con- 
siderations are discussed in the following paragraphs. 

Activities characters * In general, many of those characters have 
traditional stereotyped characteristics. Girls frequently have gushy, 
simpering voices; boys have deeper voices (even though none are meant 
to be more than eleven years of age). In one activity, two inactive 
girls are fearful of thunder; in the same activity, two active boys 
express anger in relationship to mathematical ability. Both boys and 
girls fight. The girls do not resolve the conflict and get into 
trouble with an adult"; the boys resolvt their conflict on their own. 
In a pair of similar activities, there are 3 boys and 3 girls. Three 
boys are illustrated as strongly masculine, while all three girls have 
curly hair, and one of them, holding a baseball bat incorrectly, is 
wearing a frilly "dress. , " 

Supporting non-human characters . (See Table 6 ) The three male 
non-human characters are portrayed on the audio-tapes by deep-voiced 
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Tabulation of Supporting Characters in the Story 
by Sex, Ethnicity, and Extent of Participation 



Supporting Characters 


Sex 


Ethnicity 


0 


M 


'f ^ 




Non-White 


White 




Number of 
Episodes 


Human 
















Announcers (2) 


• 












2 


Barker 


• 














•Herald 


• 














Driver 


• 










■ • 




Guards (2) 


• 








• (2) 






Nameless Boys (19) 








• (7) 


• (10) 


.(2) 


! 


Nameless Girls (28) 




• 




• (8) 


• (13) 


• (7) 




Named Boys (9) 


• 






.<4) 


• (5) 






Named Girls (7) 










• (5) 


•(2) 




Nameless Women (7)c 




e. 






• (5) 


• (2) 




Nameless Men (3) 


• 








• (2) 


•(1) 




Non-Human 
















Far Out Fish 


0 










• 




Calvin Coolcat 


• 










• 




Dragon 


• 














Piano Player 






• 






• 





? = indeterminate 

The figures in parenthesis in the body of the table indicate the number of 
characters in a g'iven group categorized by ethnicity. 
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actors. One of the three (Dragon) has v|ry low visibility. Calvin 
Coolcat is a character who is completely self-contained and cares 
nothing about feiBlings. The Far Out Fish is an abrupt advisor. 

Supporting human characters , (See Table 6.) Of the 80 supporting 
human characte-rsj 63 are children, of whom 47 (19 boys and 28 girls) 
are nameless. These nameless children have'low visibility. Those who 
are named are frequently sterotypic. For eicample, Vera the Voice 
appears to be a totally empty-headed person who is in love with her 
frivolous, caricatured "female" voice, and Susie Softshell is giggly, 
naive, and shy. . Examples of named boys are Melvin Muscleman, a well- 
meaning bumbling character who tries to create a tough-guy ' image of 
himself, and Freddy the Face, "with a smile from ear to ear and 
brightly shining eyes." Boys discuss baseball; girls talk about par- 
ties. Boys get dirty, and are active, strong, solemn and frequently 
leaders. Girls are graceful, like to be admired for their .appearance 
and hurt one another ' s feelings for petty or spiteful reasons. 

The nameless adults have very low visibility. 

There are 7 supporting characters with titles (announcers^ etc., 

as listed in Table 6), of whom 5 have speaking "Mparts • All 7 of these 
characters are males in occupations traditionaTly held by males. 

Main non-human characters . (See Table 7*) Of the 17 main non- 
human characters, 12 are Feelings. There are six pairs of Feellhgs , a 
large one and a small one for eac': the following: glad, mad, sad, 

lonely, scared, and hurt. All Feelipgs have almost equal visibility. 

I 
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Table" 7 

Tabulation of Main Charac-ters in the; Story 
- by Sex, Ethnicity, and Extent of Participation 



Main Characters 


Sex 


Ethnicity 


Number of 
Episodes 


M ^ 


F 


Non-White 


White 




Human 














Jack 


• 










20 


King of Hearts 


• 






• 




8 


\ Fast-Talking Howard 


• 




• 


• 




2 


\ Daringly Different 


• 










2 


\ Queen of Hearts 












5 


\ Marsha Mellow 




• 








'4 


\ Zookeeper 






• 






1 


X Freddy Finkletter 


• 






• 




1 


Doctor 




• 




• 




1 


**Teacher 




• 








24 


**Narrator 


• 






> 


• 


24 


\ 

Non-Human 












\ 


Mighty Glad 












? 

15 


A\ Little Glad 




• 








15 


Hopping Mad 


• 










15 


A Little Mad 












15 


So' Very Sad 




• 








15 


A ^ittle Sad 




• 








15 


Lonely A Lot 




• 








15 


A Little Lonely 












15 


Scared To Death 




• 








15 


A Little Scared 




• 








15 


Horribly Hurt 












17. 


A Little Hurt 




• 








17 


Asterisk 


• 










9 


Oopse?: 


• 




' I 
■ 1 






3 


Droop 






1 






2 


A* Alligator 


• 




1 






5 


Bessie Bluechip 




• 


i 






2 



? = indeterminate 



Characters are "outside" the story, and aie not illustrated. 
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Each is characterized by the given name^ (feeling) • Only four Feelings 
are male: Mighty Glad; A Little Lonely ;/Hqppy Mad and A Little Mad. 

Of the 5 remaining main non-hum^n characters, only one - Bessie 
Bluechip, with the characteristics of a night-club singer lacking self- 
confidence - is female. The Droop, a lonely, despondent creature, is J 
the least visible of the 4 male nain non-human characters. The OOPSEiE^^, 
a robot whose initials stand. for Only Original Person Scanner, appears 
in three story installments, and has the- characteristics of logical 
intelligence, assurance and empathy, although this- last trait causes. 

him to break down and cry when he begins to experience feelings. The 

\ ' , 

Asterisk, a professorial character, is wise, calm and sensible. The 

Altogether Alligator, is a self-assured, friendly, and cheerful extro- 
vert. ' . I 

M ain human characters . (See Table 7«) Three of fhe 9 main — ^ 
characters in the story are femald. The Queen is self-ef f acTn^g and 
although she is included in f iver.installmeuts > hor visibility is ver,y 
low. The Doctor, ajjpearing in one story insnallTD^ont , is dependent 
upon the OOPSER. Marsha Mellow, x^first appearing under the pseudo- 
nym of Thistle E. Thorn, is initiariy domineering and very self-con- 
fident, but later breaks down, cries, and is dependent upon others^for 
moral support. Marsha Mellow, appearing in four installments, is the 
most highly visible female in the program. 

The most highly visible male is Jack, a boy of about eleven, who, 
as tha story develops, learns to identify and express his feelings. 
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and demonstrates characteristics reflated to leadership, independence, 
and "empath^^ 'Jack* s father, the King, appearing in eight installments, 
•is a stereotypic middle-class professional. Daringly Different, 
yr ^ appearing in only two installments, has fairly high visibility, and 

is independent and adventurous.' Fast Talking How"^rd is a self-confident 
exti'overt* Freddy Finkletter is a slightly paler image of ^Howard. The 
?ookeeper has little depth of character until he begins to cry because 
he thinks that others ignore him. * 
Representation of Characters by Race- or Et h nicity * 

As stated earlier,, it is impossible to determine specific racial 
or ethnic characters beyond the differentiation between whites and non- 
whites. 

Of the 202 characters, 37 (18.3%) are non-white, 80 (39.6%)'are 
white, and a5 (42%) are of indeterminate race. Of the 117 deter- 
minates, 31.6% are non-white, and 68.4% are white, which indicates 
compliance with the guidelines. (See Table 5.) Of the 85 of indeter- 
minate race, 23 are non-human. This breakdown is based on illustra- 
tions ; ill of the voices on the audio cassettes sound white. 

Aucivities characters ^ Of the 92 activities characters, 17 are 
non-white, 30 are white, and 45 are indeterminate. Characteristics 
and roles are represented fairly equally across the three groups. • 

S upporting liuman characters . Of the 80 supporting human charac- 
ters, 19. (23.8%) are indeterminate. Among the 61 idei^tlf isble sup- 
porting human characters, 31.1% are , non-white and 68.9% are white. 
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(See Table 5.) Of these 80 characters, 63 are children, of whom 19 are 
non-white, 33 are white, and 13 are indeterminate. Of these children. 
14 are identified' by iiame, 10 of-- whom are white, and 4 are non-white. 
The portrayals of two of the named non-white children may be perceived 
as unfortunate: Hi Ho Silver tongue is awarded a prize for oral expres- 
sion, yet his acceptance speech is flowery, verbose, and repetitive; 
Donny - in a game - pretends to be a hippopatamus and is described as 
lazy, dirty and fat, the description being stated by ether characters 
on the tape and reinforced by the illustration. Named white children 
are referred to in the earlier discussion related to representation 
by sex. 

Although all 10 of the nameless adulti; are white, the absence of 
non-whites may be perceived as of little importance, since all 10 have 
very low visibility. 

Of the 7 titled supporting characters, 2 are white and 5 are in- 
determinate. None have high visibility. 

' Main human characters" . Only 1 of the 9 main characters is non- 
white , the zookeeper. This character has his title but no name. He 
is illustrated as over-weight, bald, round-faced, with a curly handle- 
bar moustache, and wears a uniform. He is somewhat of a nonentity un- 
til he breaks down and begins to cry because he feels himself ignored. 
The white characters are described in the earlier discussion related to 
representation by sex. 
Independent Review 

Since the first stage of the affirmative action plan for this pro- 
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gram was carried out by a white female, and since the findings • especi- 

; \ 

a].ly those related to representation by race^or ethnicity seemed to 
irdicate unfairness in representation, two independeri^t reviews were 



jcanducted. \ 

Individual Review > The following revieW report wa'^ written by a 

\ 

female black evaluator, a professional staff employee of RBS. Since 
it is relatively short and certainly relevant, it is included here in 
its entirety* 

The following procedures were used in reviewing 
the Heartsmart Adventures. I first read the affir;- 
mative action plan and the report based on the fie^.d 
test materials to determine if they matched^ I then 
read the incroduction in the Manual, listened to two 
tapes and skimmed the student materials'. 

In reviewing the affirmative action plan and the 
report based on the field test materials, I examined 
the general conventions listed on page 3 of the former 
to see if they were congruent with the latter and the 
student materials. I found that, in seneral, phrasing 
of directions were gender-free and that the illustrar 
tive examples of concepts, except when noted in the \ 
field report, were not sexually biased in terms of 
number, role or character trait. The activities seem' 
to be gender-free and should appeal to a fourth-grade! 
student population. 

The illustrations of human beings, although in- . 
- eluding persons of both sexes and racial groups, were 
monotonous . Black characters had the same facial 
features and expressions as their white counterparts. 
In fact, except for the color pigmentation given to j 
the Blacks, they could be viewed as white I ! 

In addition, only on't female (who just happened i 
to be Black) was found illustrated with short hair. 
The rest' of the females illustrated, both Black and 
white had long hair. This may reinforce the notion 
that only long and flowing hair is acceptable in our 
society. 

V The role of the zookeeper is offensive. He re- 
presents, along with the Happy Hippo, all the negative 
stereotypes about Black people. He has no name, just 
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a title; he is callous and cold, ignoring the little 
feelings (portrayed in the cartoon as little white 

beings) • He is illustrated as meek and sounds white 

on the tape, I jwquld_ no t. change-the zookeeper to a 

Bl^ck^femaleT it is already bad enough that this 

character is illustrated as a Black male." 

I find the statement regarding Hi Ho Silvertongue 
on page 9 of the TIH report* unfortunate. Why is it 
assumed that "tV^s child with such flowery verbosity 
is a foreigner to most children, and particularly to 
inner-city BlacKs-'* Is the assumption accepted by the 
authors of the report that inner-city Blacks are not 
exposed to flowery language? Is the assumption made 
that such children are not verbose or are not used to 
hearing such flowery verbosity? Or is it that the 
authors of the report have never heard Black children^ 
playing the dozens, or doing hand-jive or jumping rop^" 
to oral rhythms, which is flowery verbosity from a 
different cultural orientation, I recommend that the 
language of the report be changed. 

The general purpose of this program, emphasizing 
as it does that people are rewarded for expressing 
their feelings, offers potential conflict for children 
from nonmainstream communities, especially when the 
expressions of those feelings are made, to outsiders 
(the school commur*iry of orientation as „ represented by 
the program and its teachers.) This expression of 
feelings is fraught with danger because it can be seen 
as threatening to the child's functioning within his 
home and community if the teacher is not overly sensi- 
tive to the God-like role he or she is implementing. 
By not allowing parents to observe the process being 
taught, an image is created of the program intervening 
in a- relationship established since early -childhood 
(between parent and child) which may not be understood 
or appreciated by the mainstream culture. 

My basic objection to the program does not lie 
with the inadequacy of the illustrations or the stereo- 
t.\ jed roles portrayed by minority group members but 
V- 1 th the underlying value orientation of the program 
'• that in order for children to become functioning 

man beings they must learn how to communicate their 
•r.nermosty feelings and thoughts t^ others. This 
could be quite detrimental to young Blacks growing up 
in a hostile racist country such as ours. 



*Refers to an early draft of the preceding pages of this section. 




G roup Review . The third stage of the affirmative action plan 
called for the assistance of non-white persons. Seven Black empljyoes 
of RBS' — 3 male and A female — one of whom was professional stati, 3 
executive level support staff, and 3 general support staff, agreed to 
participate in an informal review. 

Each participant read the Information Pages (Appendix A) before 
attending a meeting. The meeting was chaired by the person who con- 
ducted the first stage of the affirmative action plan, but she did not 
state .opinions nor sh^re her findings* Participants examined Heartsmart 
materials and listened to excerpts , from three audio tapes* Subsequent 
discussion was guided by a series of questions. 

Participants were inter'='sted in the program and generally suppor- 
' tive of its goals* However, they expressed concern with possible value 

conflict and suggested that children be taught that the expression of 

\ 

feelings is natural and not wronr, but that i.. is rot always rewarded, 
and one must be sensitive to the situation in which such behavior is 
appropriate and ?ven possible* 

There were alro criticisms of black characterization but part.ici- 
pants of. the meeting unanimously agreed that adequate revisions of black 
characterization could be made within the existing format of the program* 
A summary of their recommendations follows: 

General: 

• Give blacks and females greater visibility* 

• Give members of these two groups stronger characterization, 
and elimitiate stereotyped roles. 
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• Give "identity to at least one black female, 

• Re~do art work so that blacks look black. 
9 Include black actors to play black roles. 

• Include other ethnic group members. 
Main Characters ; 

• Have one more black - perhaps one of the TV or rad "':^rsons. 

• Give one female some spirit - perhaps Daringly Dif f£;"^?u could 
be female. 

• Do not have the zookeeper black; this character is not * 
or admirable. 

Supporting Characters : • . 

• Make more of the identified characters black and female. 

• Get rid of gushy female voices. 

0 Change images within the animal ;j;aine. Have both boys and girl 
choose animals with "feminine" and "masculine" characteristics 

• Take out the girls' disgust with dirt. 

• Change the Happy Hippo - he should not be black. 

• Change Hi Ho's speech and name: H-i*" speech should be expres- 
sive but realistic; a black perse, should record it. 

• Use' more female voices. 
S ummary and Recommendations 

The quantitative data and opinions presented indicate that repre- 
sentation of. characters by sex, race or ethnicity, and behavioral 
characteristics is socially unfair in some cases, and affirmative ac- 
tion should "be taken to ensure a better balance. 

In order to achieve social fairness in all of the foregoing areas. 
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illustrations of non-white characters should be redrawn or newly cre- 
ated; some characters should change sex or ethnicity; schie characters 
should be conceptually revised by changing dominant behav jral charac- 
teristics; and- all stereotypic characters should be carefully evaluated 
in order to determine if their representation is contrary to the intent 
of the ''affirmative action guidelines. 

Most of the recommendations made by individual and group reviewers 
should be implemented. . . 

(Revisions made are described in .Chapter Six. ) 

III. PROGRAM IMPLEMENTATION 

Information pertaining to program implementation refers to the ex- 
rent to which suggested implementation procedures were follov;ed at ^:ach 
o2 the individual sites. Teachers were instructed on these desired ■ 
implementation procedures by the teacher's manual,^ and also during the 
teacher training session and at informal talks during sire visits. Iv- 
formation for this section was derived primarily from observational 
data and to a lesser extent from teacher and student self-report mat^^ri- 
'•'Ls. 

\n Chapter Two of this report program implementation is described 
as presented in the teacher's manual. However, for the field test, it 
was necessary to modify or clarify some of the directions given in the 
manual. For example, although the manual allows for program participa- 
tion by substitute teachers, it was agr*^ed that substitute teachers 
should not participate in the field test. Another change related to 
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scheduling; ficUd tert teachers were asked to complete two to three 
Heartsniart xessoms and not one per week as suggested in the manual. 

In order to clarify the contents of this section it is necessary 
to consider a variety of factors and variables. The following assump- 
tion is made: 

0 

• Given the program materials and written and oral directions as to how 
those ii:aterials should be used, the teacher interprets the directions 
a^'id develops dispositions which lead to covert or overt decisions re- 
lating to the nature and extent of deletions and /or additions the 
teacher makes and the degree of emphasis the teacher puts upon each 

direction. Thus the teacher influences student achievement and program 

♦ 

effectiveness on several variables, including: time spent' on tasks; 

cognitive and affective understanding of and disposition toward the 

program and parts of the program; disposition toward the students; and 

-I - * / 

. . " f< 

nature and extent of interaction between and among the teacher, the 

students, and parts or elements of the program. 

Although this assumption is highly relevant in attempting to deter- 
mine the impacts of differences in program implementation among plasses, 
it must be treated ccautiously, because the field test was not designed 
to evaluate teachers, and because observational data of classroom acti- 
vities are somewhat subjective. However, the assumption may be used as 
a conceptual framework for this section, and every effort is made to 
be as objective as possible in interpreting observational data, most of 
which are supported by 'teachers' self-reports. 
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Materials . 

All teachers received identical program materials within the same 
two week period approxfimately 10 days before the teacher training ses- 
sion, and 4 weeks before the program was used in the classrooms. With 
.the exception of the teacher's manual, all materials were used as in- 
tended. The administrative burden on the teaCi^er to check and distri- 
bute the loose student journal pages for each lesson was considerable, 
but did not cause major problems. 

The^exception referred to above, the teacher's manual, involves 
directions, discussedl below. 
Directions 

All teachers were given identical- initial directions from two 
sources; the teacher's manual and the teacher training session. During 
the latter, cer -^in points given in the manual were clarified so that 
teachers understood that field test directions over-ruled some manual 
directions, such as those described earlier in this section. 

There were three categories of directions. 
« Imperative directions riBlated to scheduling, sequence and complete- 
ness of implementation of structured tapf-led lessons and Heart to 
Heart talks. , " 

• Guiding directions related to the teacher's role during all lessons, 
especially the Heart to Heart talks, and to general administrative 
tasks.. 

• Discretionary directions related to the use o^ optional teacher-led 
activities and songs'. 
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Supposedly, the teacher's manual and training session provided 
sufficient imperative, guiding, dVd/or discretionary directives so that 
the program would be implemented as intended. To a large extent this 
was so. However, there were variations in interpretation by the teach- 
ers, and in the ways in which the teachers' dispositions (toward the' 
program, the directions, and the students) affected program implementa- 
tion. 

Table 8 summarizes the directions given and the implementation of 
these directions for each site. In general, all sites followed the 
suggested implementation procedures very closely. All sites ad)hinis- 
tered the program at l^ast twice a week, presenting all lessons com- 
pletely and in the given sequence, used a fair to large number of op- 
tional activities to supplement instruction, and played' the Heartsmart 
songs at appropriate f'mes. 

Teachers from all but one site (E) had relatively little difficulty 
finding an acoustically optimal, or at least satisfactory, location for 
the tape recorder in their, classrooms. The classroom at site E did not 
lend itself well to a taped presentation, and, consequently, the teacher 
reported that most students could not hear the contents of the tape as 

clearly as possible. No sites reported having had outstanding mechani- 

/-^^ ^ 
cal or operational difficulties with the tape recorders. 

.■^ 

For the most part, teachers from all sites ensured that there was 
a minimum number of distractions during the tape, presentations. The 
distractions that did occur were relatively minor (e.g., fire dril]/S, 

o 
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I Table 8 

9 

Suggested and, Actual Iciplenent'atlon of Prograo During Field Test 



Suggested Impleoentation - Actual ImpleDentatlon At Site 



GENERAL 

2 to 3 lessons are conpleted per week. I 


A 

■ 3 per «eek 


B 

2 per week 


C 

2 per week 


D 

2 per week 


E 

2 per week 


F 

2 per week 


G 

L per weeK 


Optional activities are used at 

teacher's discretion.* D 


8 

3 

3 • 


' 9 

7 

7 


4 

3' 

2 


7 

'■3 

5 


5 

0 

0 


8 

3 

0 , 


8 

6 

4 


Heattsmtt songs are used at 

teacher's discretion. D 


All sites used /leartsmart songs to some degree. Sites A, B/'D, F and C used the songs 
extensively. " ■ • 


TAPED* LESSONS 
Teacher positions tape recorder in 
room for best acoustic result. G 


Site E was the only sire to have rep'^ated problems with acoustics. ^ 


Teacher allows for minlinuiii dis- 

tractions while tape Is being 

played, C 


No outstanding distractions were reported or observed for any sites. Minor distractions 
included fire drills^ voices over the intercom and disruptions by students. 


Teacher acts as a "facilitator" 

during tape-led lessons. G 


Teachers from all sites were responfllve to student needs and offered clarification oi 
direcclons,^ stopping the tape when necessary. Interferences were kept to a ninlniuiii. 


■ tape 


HEART-TO-HEART TALKS 
Talks take approximtely 23 nlnutes 
to complete . G 


35 icin. 


35 nln. 


30 nln.- 


34 nln. 


35 nln. 


40 Bin. 


0 

40 uin. , 


f ^ 1 ' — 

All main points and questions covered. I 


Yes| 


Yes 


Yes : 
-^s, 


Yes . 


Yes 


Yes 


Yes 


Students sit in a circular formation. I 


NO) Ss sat in . 
usual class 
arrangetaent»+ 


Yes 


Yes 


' No, Ss • 
clustered 
around T. 


No, Ss 
clustered 
artfUnd T. 


Yes, at times; 
otherwise clust- 
ered around T. 


I 

Yes 


Teacher relates rules for talks at 
beginning of discussion* > i 


Yes 


Yes 


Yes 


Yes 


Yes 


yIs 


Yes 


NUeacher acts as a discussion leader 
and facilitator during the talk. G 


' over-extended 
role of 
facilitator 


Yes, 


Yes 


Yes 


Yes 


Yes' 


overly 
cognitive 
role" 



I = imperative directions *fl8""8 reported are the number' of optional activities • 

G = guiding directions "sed for Units I, II, aild III, starting with Unit I in 

, D = discretionary directions the upper left-hand corner. 

, **rigure8 reported are averages . 

■I . +De8k«i were in rows or in groups of three and four. " 
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voices over the intercom, and brief disruptions by students) » Teachers 
also followed the suggested guidelines and acted as facilitators during 
the presentation of the tapes. Whether they circulated around the room, 
or sat at their desks or at empty pupils* desks, the teachers were very 
attentive of the tape and, as recommended, were responsive to' student 
needs with respect to direction clarification. 

The Heart to Heart talks were completed as scheduled, with teach- 
ers covering the given content and using more time than had been 
planned. Although not all sites conformed to the suggested circular 
formation, ail. but one site used some kind of informal *'group-oriented" 
arrangement. All teachers were conscientious about relating and/or ' ^ 
eliciting the riMes of the Heart to Heart talks before the beginning of 
the discussions. The time spent on these rules, as well ai> the' for- 
mality of the presentation, waned a little as the program progressed 
and students became more familiar with the format of the talks. 

Interpretation and Disposition 

/ 

The preceding disci. 'jsion indicates that all teac^v^rs understood 
the directions given, and/follow^ed those dir'Xtions fairly closely. 

Hovever, since there were "open" areas for >.hich no directions were 

i 

given> variation among classes became apparent to observers. It was 
decided that no additional directions should be given -to teachers unless 
a variation was considered harmful- No variation was so considered. 
In some instances teachers asked program staff if such variations as 
they had introduced were acceptable. The standard answer given was that 
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if the teacher considered the variation desirable, and if it did not 

(. 

contradict given directions, it was acceptable. 

For the most part variations seemed to be dependent upon the 
teachers' dispositions toward the program, and occurred imniediately 
after a tape-led lesson, during Heart to Heart talks, and in the option- 
al^ teacher-led activities. 

Tape-led Lesson s . As' stated before, all teachers implemented the 

tape-led lessons as directed. Some variation was observed during the 

part of the lekson in which the teacher on the tape directed students 

in a pencil arid paper* activity. In some classes the students did not 

hear or did not 'understand the taped directions- so that the classroom 

teacher" needed 'to stop the tape, and explain the directions. In most 

classes such occurrences were rare, but in oni: class the occur-rfences 

'i . 
were more freiijuent. 

The major., unanticipated variation to tape-led lessons occurred 

after the tap^ had ended. The only (discretionary) direction given to 

1 ■ • 

teachers stated that optional activities could be conducted immediately 

\ 

alter a tape-led lesson. Sometimes* teachers <^id conduct an optional 

! ■ ■ 

activity^ More frequently, .teachers chose to summarize the lesson and 

f I -9 . . ; 

conduct a discussion. These conclus ionary activities yere known infor- 
mally as "wrap-ups." • ' ' : 

Table 9 |ummarizes the average "wrap-up" for .^ach site in terms /of 
time spent, frequency and content. In examining , this table it is im^ 

i ' ■ " - •■ ' ■ ^ ■ / 

portant to remember that no directions were ^ive'n for "wrap-ups ," /and 
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; Table 9 

"Wrap-ups*' Following Tape-led Lessons 

\ 

s ■ 



SITE // 


AVERAGE 
TIME SPEOT 


FREQUENCY- 


USUAL CONTENT 


ri 

A U 


:30 minutes 


Always 


Comprehensive;' summary; 
discussion; optional 
activities _ • 


B 


5-10 minutes 


Almost 
Always 


Limited : j- summary ; 
discussion 




Varied, usually 
. less than 15 
minuses. Some- 
times none at 
all. 


Varied 


Varied: J summar)r>; 
discussion; occasional 
optional activities - 


D ' 


Varied, usually 
15-30 minutes 


Always 


Comprehensive : summary ; 
discussion; optional 
' activities 


E 


None, ' 


None 


None 


F 


Varie. / v -30 
, minute- 


Always 


Varied: summary; 
discussion; occasional 
optional activities 


G 

5 


5-10 minutes 

i i 


Infrequent 


Limited: summary; 
. discussion 
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that some teachers conducted optional teacher-led activities separately 
fron) the structured lessons. 

In general, "wrap-ups" consisted of summaries which were similar 
to the tape- led reviews^ and brief discussions which were often based 
on discussion points given in ti.e optional activities suggestions. 

■ Since^ these "wrap-ups" were unanticipated by program staff and 
therefore were not dealt with by directives, ic could have been possible 
for' teachers to influence the desired program .imp ^ "t negatively by in- 
troducing conflicting concepts. Initially this was o real concern, re- 
lated primarily to potential harm to the students. However, since 
teachers understood the concepts of the program and seemed disposed to 
agree with the goals and objectives, they did not introduce conflicting 
concepts. Observational data indicate that some form of "wrap-up" was 
beneficial, both in terms of allowing for active participation of the 
teacher and students (in an otherwise somewhat passive -learning situa- 
tion),,and in allowing for clarification and pradtice of program con- 
cepts. • * . ^ i 
■J 

Heart to Heart talks . As explained in Chapter Two, there were six 
teacher- led Heart to Heart talks interspersed among the twenty-four 
tape-led lessons. Imperative directions included discussion rules for 
the students, seating arrangements, content in the form of main 'points 
and questions, and a strong implication that all main points and ques- 
tions should be covered. Guiding directions included time (20-25 min- 
utes), and "rules" for the teacher: repeat and reflect students' state- 
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ments; give support; vary questions; and keep discussion focused. Dis- 
cretionary directions suggested that anger could be discussed, focusing 
on generation of ideas of acceptable ways of expressing anger. Also, 
in the "trouble-shooting" section of the manual, some suggestions were 
given for shortening or lengthening the" talks, for increasing student 
participatioh, or for dealing with discipline problems. 

It becarfe apparent at the teacher training session, after x ers 
had read the manual and discussed the Heart to Heart talks with the 
developer, that some teachers were not confident in their comprehension 
of what was to be accomplished in the talks, and were disposed to be 
wary of anticipated student reactions and/or nature of participation in 
the talks. 

As the field test was intended as a "hands-off" evaluation situa- 
tion, it was decided that observers would offer further, guidance only if 
a teacher asked directly for assistance and/or if -^the observer judged 
that students were being harmed- by the content or implementation of the 
talks. Almost no such guidance was found to bef necessary. 

As shown in Table 8, two imperative ' directions , followed by all 
teachers, relating to Heart to Heart talks were those relating to the st 
dents' discussion rules, and to covering all main points and questions. 
The guiding direction relating to time was probably unreasonable if all 
content was to be covered and all students ^n.:ourased to participate. 
The guiding and discretionary directions relating to., the teacher's role 
were the most problematic. 
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During the Heart to Heart talks teachers were or felt free to 
develop their own roles around the "skeleton" model offered in the 
guidelines, A wide range of teacher roles was thus possible, albeit 
rtot necess'arily interi^ed. Most teachers dealt with the Heart to Heart 
talks appropriately, acting as discussion leaders and facilitators, 
articulating and also elaborating upon the main points of the Heart to 
Heart talks, and allowing for perj?onalization of the concepts. One 
teacher, however, followed the Heart to Heart "script" very closely, 
elaborated very little, and made the talks^ assume the form of cognitive 
exercises, in which the students practiced //eartsmarfc-related concepts, 
yet did not extensively relate these concepts to their own lives. In 
contrast, a teacher at another site greatly extended the role of facili 
tator and functioned in some ways as a "therapist," iocusing on solving 
the children's personal problems in addition to providing opportunities 
in which the students' feelings — as they related to their ^problems — 
could be aired and discussed. Many teachers conducted^additional Heart 
to Heart talks wherein students discusser classroom problems. Private 
Heart to Heart talks were also requested by students at several sites. 

Observational data indicated that all teachers took the Heart to 
Heart talks seriously, but each teachet was disposed to create a differ 
ent atmosphere from the others. For instance, although all teachers 
were receptive toward students' contributions to the discussion, some 
teachers encouraged informality, while others maintained a traditional, 
s^.j'^what cognitive approach to the lesson concepts. All teachers 
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would ask a question to clarify the point made or to lead the student 
closer to the focal point. Occasionally a teacher would tend to ser- 
monize. These, and other differences are subtle and seemingly minor. 
However, the way in which a teacher conducted a Heart to Heart talk 
clearly affected the nature of the interaction among students during 
the talks, and possibly had an impact upon the overall effect of the 

o 

program. 

Because of the lack of definitive directives to the teachers about 
the Heart to Heart talks, the possibility of potential harm to the stu- 
dents was fairly high. For instance, a teacher (without training in 
clinical psychology) playing the role of diagnostician or therapist 
could have manipulated students* f'eelings, possibly causing psycho logi- 
cal, harm. Also, a teacher with strong views on social desirability 
of certain behaviors copld have caused psychological harm by 
advising students that certain related feelings and behaviors were 

''wrong." However, although program staff observers were concerned 

♦> ** 

about such possibilities, data indicate that most teachers did nOt mani 
pulate or judge their students in these potentially harmful ways. 
Te.achers did take it upon themselves to "localize" the contents of the 
Heart to Heart talks, by emphasizing some points and skimming others. 
For instance, at least one teacher responded to a student's direct 
question by stating that the child should, "Do what's best for you. 
Something that works for Jack in the story might not work for you at 
home." Such a statement x^as not in conflict with the program, but was 
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an example' of the ways in which teachers judged thejconcepts as they 
applied to their students. 

Optional activities . The discretionary directive encouraged 
teachers to use optional activities, but teachers were free to choose 
to use none at all. ^ 

As shown in Table 8 all teachers used some optipnal activities, 
either as part of the "wrap-up" or at another time. Several teachers 
made up their own activities in which ^leartsmart concepts were linked 
with other curricula. Some of the teachers modified or expanded acti- 
vities suggested in the manual. Some activities were used as given. 

In general, the' strongest factor influencing the selection-and use 
of optional activities appeared to be the teachers' dispositions toward 
the students. Thus, a teacher disposed to believe that her students 
needed "active" learning would choose or generate activities . involving 
students in improvised dramatization. Another teacher disposed to be- 
lieve that her students needed practice in written expression would 
assign some form of written Wurk. These two examples serve as illustra- 
tions only, and are not comprehensive. 

In referring to the optional activities section of Table 8, it is 
important to note that the figures given tor each site for each unit of, 
the program relat,e to frequency, not quantity. Thus, for instance, at 
site A in. Unit I the teacher reported using optional activities after 
8 of the 12 lessons in that unit. This means that at least 8 activities 
were used; it could mean that more than 8 were used. 
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Teachers were asked to. identify and to record optional activities 
used. However, only three such schedules are available. 

At school B the teacher reported using a total of 63 optional 
activities, all of which were suggested in the manual. Time taken for 
each activity ranged from 10 to 30 minutes. When discussion points 
were suggested in the manual, the teacher nearly always used them. Most 
other activities selected required students to write .^r draw. 

At schc il C the teacher reported using a total of 15 optional 
activities, allowing" from 5 to 50 minutes for each activity. The 
teacher used all the handout pages given in the manual, some of the 
given discussion points, the songs, md a few pencil and paper activities 

At school G the teacher report*ed using a total of 3? optional 
activities, allowing 10 to 45 minutes for each activity. Most of the 
selected activities required students to write or draw. 

Observational data indicated that at schools A and D many optional 
activities had been used since a great deal of program-related student 
work was on display. At school F the teacher initiated several progrant- 
related activities of her oym in addition to some suggested in the manu-- 
al. At school E students were involved in very few optional activities. 
Impact Upon Achievement and Effectiveness 

As stated in the assumption presented near the beginning of this 

section, the teachers' interpretation of directions and disposition to- 

<? ■ ■ . 

ward the^..gceagXam, students influence implementation which in 



turn^ljaluei^^isr^tj^d^^ achievement and overall program effectiveness. 
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Data are not available on all variables for all /^^os. However, 
some generalizations may be made. 

• Time cn ^ask: School A spent the most time on the program, followed 
ill ranked order by schools D, B, F/G, C and E. 

• Cognitive comprehension: All teachers demonstrated complete, accurate 
comprehension of the program and the tasks to be performed by teachers 
and students. 

•J 

• Disposition: Teachers demonstrated very little differences in their 
dispositions toward |:he program (see Section IV of this chapter). 
Dispositional behaviors were more apparent in teaching styles, parti- 
cularly during the Heart to Heart talks, and ranged from an open 

^ _ . . 

warmth showing genuine interest in the students to a somewhat closed 
coolness close to' boredom. 

• Interaction: In the Heart to Heart talks the traditional student to 
teacher interaction was common in all schools although one teacher 
encouraged student to student interaction. In the tape- led lessons of 
Units II and III students were required to interact within their 
small groups. This interaction- was appropriate and positive in all 

, schools. However, when a group leader was over-bearing or ineffective, 
the behavior of members was influenced, resulting either in quarrels or 
in lack of task orientation. In all schools, the teacher^ intervened 
in group interaction only when it was apparent that sych intervention 
was necessary. « . 

Without reference to other data it would seem that the way in 
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which each teacher implemented the progra.n had some influence on the 
impact of ^the program. In. as much as all imp^erative directions were 
followed relatively closely, all guiding directions were interpreted 
fairly accurately,, and all discretionary directions were duly considered, 
it can be said that all essential parts of the program were implemented' , 

as intended and the variations in implementation should have no negative 

effects on achievement and effectiveness. 

IV. PROGRAM APPEAL AND ACCEPTABILITY 

Results relating to appeal and acceptability reflect the reactions 
of the studo.nts and teachers to the program materials and cpurse of 
instruction. Reactions are reported iu terms of ^,the following cate- 

gories: student reactions to the program; and ^teacher reactions to 

' . . <. 

the program. 

Student Reactions to the Program 

Following each lesson five students in each class completed appro- 
priate opinion surveys. All students in each class completed the last 
opinion survey relating to Heart to Heart talks, and all students com- 
pleted the survey relating to groups. 

- Tape-led Lessons . ' The results of the analysis of responses given 
by students to items asking them to rate the taped storits and games 
according to appeal are presented in Figure 3. Students could select 
responses from four alternatives: (1) a lot, (2) some, (3) a little, ; 
and (4) not at all. Only the percents of students who reported having 
liked the stories and games either "a ^Lot" or ''some" (the two most positive 
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•—"Taped St9ries 
O- — « Taped Gaines 

Figure 3: Percent of students reporting to have liked 
the taped stories and games either "a lot" 
or "some" shown as a function of lessons. 

y ■ 
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dimensions of th^ scale) are reported. Presentation of these positive 
reactions was considered to be a good indication of program acceptabil- 
ity.; ' • " . 

Student reactions to the stories and games were very similar. The 
mean percentage of students having liked the stories either "a lot" or 
"some" for all three units was 80.4 with, a range of 61.7% to 96.6%. 
Ancilyzed by unit, the means were 86.^8%, 74.3%, and 76.r9% for Units I,^' 
II andNjII, respectively. The mean percent of students having liked 
the taped games for all three units, was 77.2 with a range of 58.8% to 
96.1%. Means for each-unit were 82.5%^ 71.7%, and 74.5%, respectively. 
Positive reaction to the stories and games as reflected in the percent- 
ages were, thus, initially very high -(theil: highest), gradually declined 
until reaching their lowest levels toward the middle and end of Unit II, 
and then increased again in the later lessons of the course. 

Figure. 4 presents the results of the analysis of student ratings 
of the taped games with respect to difficulty. Students could select 
from two alternatives: (1) easy and (2) hard. The'mean percent of stu- 
dents, finding the games to be easy was 94.7 with a range of 78.7% to 
100%. The individu.'.,l unit meajfis were 94.5%, ^2. 8% and 97.5%, respec- 
tively. ■ - 

Analysis of the comments to an open-t:nded question asking the stu- 
de^ntF. to write in any thing \hey wanted to say about the game of the day 
yielded three "categories or characteristic patterns of responses. Stu- 
dents expressed : 
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TAPED LESSONS, UNIT I 



100 




TAPED LESSONS, UNIT II 




23 2A 26 27 ' 28 
TAPED LESSONS, UNIT III.. 



Figure 4: Percent of Students rating taped games as 
"easy" and "hard" shown as a function of 
lessons by Unit. 
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1. an adamant l.iking for the games; 

2. an appreciation of the fact" that the games offered' them 
opportunities in which to. learn to express, their feelingis; 
and \ 

3. a desire for the games to have been more exciting and chal- 
* lenging (less easy). 

Samples of students' responses from these three categories are 

presented below. Samples are representative' of all sites. 

Representative Sample of Student Responses to: "Write in 
anything you want to say about the game today." 

Xiking for the games: 

- .They were really easy, fun and neat. 
"^"^ - I^iked'it a lot. 

- I like the game and the work pages because it's lots of fun 
and I like coloring and drawing. 

- The "game was filti. 

- It was fun. It really made me happy and I'm glad about the 
whole thing. ^ . 

- It was fun'^and diffe'rent. - 

- I think it was great. 

- I like it a lot. ' It is fun. I am glad we take the program. 
Appreciation for the* opportunity- to learn how '"to express their, feelings 

- I liked when we talked about our feelings. 

- It was a fun thing to think about your feelings. 

- It told a lot about people and feelings. It was fun. 

< 

- 'I think 'it will help express your-'f eelings . 

, »^ • 

- It was teaching me more feelii^gs than I thoxight , and tHank you 
for^giving us the Heartsmart Adventures* 

- L like it very nach and it is fun and helps me. 

- I want to thank the people who invented Heartsmart. It's a 
good way to 'find your feelings. ^ 

- If was fun and helpful to some problems. 

' e 
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Desire for the games to be more exciting and challenging. 

- It was a little boring. It could be more interesting. 
^ , - It could be improved by making it more exciting. 

I think that the games were easy but still fun. 
It was a little too easy but still fun. 

- The game was alright. It was a little boring but it was fun. 

- The g.ames are too easy. 

- Can't you make it harder? \ 

\' 

Heart to Heart Talks . The results of, the analysis of responses 
given by students to an. item asking them to rate the Heart to Heart 
talks with respect to appeal are presented in Figure 5 Students could 
select responses from four alternatives: (I) a lot, (2) some, (3) a 
little, and (4) not at all. Again, only the percentages of students who 
reported having liked the Heart to Heart talks either "a lot" or "some" 
(the two most positive dimensions of the scale) are presented. The mean 
percent of students who liked the talks either "a lot" or "some" was 
82.3 with- a range^ of 63.6% to 97.0%. 

^The results of analysis of a second item asking the students to 
rate the individual Heart to Heart discussions with respect to "how 
interesting" they were are presented in Figure 6. Stu 'ents could select 
responses from four alternatives: (1) very interesting, (2) a .little 
interesting, (3) not too interesting, and (4) not at all interesting. 
The mean percent of students rating the contents within the two most 
positive diiffensions (very interesting and 3 little interesting) was' 
85.1 with a range of 69.6% Co 90. 6r^. The appeal of the Heart to Heart 
talks for the students with respect 'to both "liking" and "interest" was 
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Figure 5: Percent of students reporting to have 
liked the Heart to Heart talks "a lot" 
or "some" shown as a function of the 
talks. 
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Figure 6: Percent of students finding the content 
of the Heart to Heart talks to be "very 
interesting" or "a little interesting" 
shown as a function of the talks, 
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thus initially very high, dropped off considerably until reaching the 
lowest point in Lesson 21, and then increased rather sharply for Lessons 
25 and 29. It should be noted, however, that figures for^Lesson 21 
never fell below 60% for "liking" and 70% for "interest." 

Students were also asked to ^report their ^'strongest" feeling during 
the Heart to Heart talks. The results of the analysis of these responses 
are presented in Table 10. Students could select from six responses: 
"I was happy," "I was mad," "I was sad," "I was shy," "I felt embarrass- 
ed," and "I felt like laughing," Students could also write in their 

/■ 

own responses. The mean percent of students having felt happy during 
the Heart to Heart talks was 52.3, which is by far the most frequently 
reported feeling. The mean percents of students having felt mad, sad, 
shy, embarrassed, and "like laughing" were 7.5, 2.4, 6.9, 5.8 and 15.2, 
respectively. Students also wrote in a number of responses. These were 
categorized if they appeared for more than one talk. (The computed per- 
cent figures took all responses — given and offered — into account.) 
On an average, 3.2% of the students wrote in that they felt "bored" dur- 
ing the Heart to Heart talks, 1.3% felt "interested," .73% felt "weird," 
and .73% felt "glad." Responses given for only one of the Heart to 
Heart talks are listed in the* right hand column of Table 11. These 
included such feelings as "normal," "scared," "surprised," "bad," 
"sleepy," "hurt," "like not paying attention,", and "like helping a per- 
son." Lessons 21 and 25 had the least number of happy feelings reported 
In addition, students were asked if they wanted to talk more during 



96 

111 



Table 10 



Students' Strongest Feelings During Individual Heart to Heart Talks in Percent. 



Heart to Heart Talk 




6 

12 
17 

21 

25 

29 



32 

37 
26 

33 

31 

71 

230 



59.4 

56.7 
61.6 

36.5 

41.9 

57.7 

52.3 



0.0 

.7.7 

15.2 
12.9 
5.6 

7.5 



0.0 

5.4 
0.0 

3.0 

3.2 

2.8 

2.4 



6.3 

0.0 

I. 1 

6.0 
9.8 

II. 2 

,6.9 



9.6 

2.7 
7.7 

6.0 

6.4 

2.8 

5.8 



12.7 

29.8 
0.0 

24.3 

9.8 

14.0,., 

15.2 



2.7 
7.7 

6.0 

3.2 

0.0 

3.2 



0.0 

2.7 
0.0 

0.0 

3."2 

1.4 

1.3 



3.0 

0.0 
0.0 

0.0 

0.0 

1.4 

.73 



0.0 

0.0 
0.0 

0.0 

3.0 

1.4 

.73 



normal (3.0) 
scared (3.0) 



surprised (3.8) 
not pay att. (3.8) 

"felt like helping 
that person" (3.0) 

bad (3.2) 
sleepy (3.2) 

hurt (2.8) 



4.3 



the Heart to Heart talks. The results, of the analysis of the responses 
given to this item are presented in Table 11. On the average, 66.3% of 
the students had wanted to talk more during the talks, with 24% having 
wanted to talk a lot more and 42.3% having wanted to talk ''a little 
' more . 

The responses given by the students to an open-ended question ask- 
ing them to write in anything they wanted to say about the particular 
day's Heart to Heart talk were rather homogeneous. For the most part, 
students expressed a liking for the talks >\ commenting that they were . 



- It was fun, and interesting. 

- I like it and it is fun. 

- I really got a load off my chest, 

- Well, I like Heartsmart because I think people should know my 
feelings. 

- I like the Heart to Heart talks because they say a lot of good 
questions . 

- I like sharing my feelings with other people. 

- It gave me the impression of .what different disagreements 
people have. 

- I thought this lesson helped me with friendship. 

- I really like it because it is interesting. 

- 1 really like to say my feelings to everyone in the classroom. 
Group Work . The results of the analysis of responses given by 

students at the end of the program to items ^measuring feelings with 
respect to the group experiences offered by the course are presented in 



\ 

fun and allowed them to express and listen to feelings. 



A sample of student comments is offered below. 



- It was very nice to hear other people's feelings. 
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Table 12. Each of the quadrants of the table presents slightly diffejer- 

ft * 

ent information. Part I presents the percents of students who were 
leaders at some time during the program; Part II shows student prefer- 
ences for being either a leader or a regular group looper (member) ; and 
Parts III and IV present student feelings with respect to individual 
and group work, respectively. 

Sixty-eight percent of all students responding to this group ques- 
tionnaire were leaders at one time or another during the course.* Var- 
iation among sites was apparent. At Site E, -for instance, 92% of the 
students in the . class were leaders, whereas at Site D, only. 45% of the 
students in the class were leaders, and at Site F, only 51% were lead- 
ers. Looking at the percents in Part II, of the students who ''were 
leaders, a majority (56.8%) responded that having been leader was better 
tfhan having been a regular looper. The leaders of all sites showed 
this preference for leadership, with the students from Sites A and E 
reporting the greatest preference! 

Looking at the total figures for Parts III and TV, no clear prefer- 
ence for individual or group work was evident. Students seemed to like 
working in groups and by themselves equally well. Some variation among 
sites was apparent, however, especially with respect to the appeal of 
the group work.. The students at Site D seemed to enjoy group work a 

*The teacher's manual informed teachers that there were opportunities 
for four people to be leaders for each group. The first leader was 
to be selected by the teacher, and the others elected within groups 
by students. Students were allowed to reelect leaders. There were 
5 or 6 groups in each class. 
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Table 12 



Student Reactions to Group Work in Percent by School 




I 

II 
III 
IV 



Percent of Ss having been leaders. 
Preference for being a leader or a looper. 
Appeal of Individual Work. 
Appeal of Group Work. 



+No group survey forms were returned from this site. 



ERIC 



117 



great deal, 80% reporting to have liked working in a group '> lot." In 
contrast, the students from Site E showed fhe least preference for 
group work, only 26% reporting to have liked working in groups "a lot." 
Relatively small percents (approximately 15% for both group and individ- 
ual) of students reported having not liked working in groups or by 
themselves "at all." It is unclear, however, whether these two groups 
were different sets of students or the same sets of students expressing 

V 

a dissatisfaction with the entire course. 

Samples of the responses given by students to two open-ended items 
asking them . to write in what they liked best and least about their 
groups are presented in the following page&. Responses for. the "liked 
best" icem fell into three categories. Students expressed a liking f-or 
the groups for the reasons presented below. 

1. Their friends or people they liked were in them. 

2. They liked having the chance to be leader. 

3. They ^liked being together, cooperating, and sharing. 

Representative Sample of Student Responses to: "Wh at was 
the thing you liked best about your looper group?" 

1. Having friends in the group. ^ 

- I like the people best. 

- We had people that were funny. \. ^ 

- I had friends in it. 

- I met nice people. 

- I was with my friends. 

- I had my friend ^n my group. 

- I. got to work with my friends. 
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- The group was my best ftiends. 

- They were funny and nice. 

2. Being: a leader in the group. 

- Being leader in the group. 

- Being a leader. 

- To be the leader. - / 

- I like when I was the leader. : 

^ Being leader so I could say, "be cooperative." 

3. Being together, cooperating and sharing. 

You could ask things without yelling out. ^ 

- We could share things. 

- Being together. 

- The cooperation and people. u ' . 

- When, people felt good. 

- Everyone cooperated. 

- I liked speaking out about my feelings to my group.. 

- We were all good natured and didn't fight. 

- I 'liked the troubles we helped. 

- We told our feelings. * 

- The things we did together. 

- Everyone was^ nice. 

- ^You got' to know people better. 

Responses for the "liked least" ifem also fell into three cate- 
gories. Students commented that the things they did not like about 

their groups were: " < ^ , 

■ \ 

1. having one or more persons in the group they did not like, 

2. too much general noisemaking and disruption within the group 

3. being exposed to some mildly negative situations. 
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Representative Sample of Student Responses to:' ^'What was 
the thing you liked least about your looper group?^' 

K Having disliked people in the group. 

- Some of the people. 

- I didn't like someone in my group. - 

- Being without some of my friends. .c 

- I disliked such and such a person. 

- I didn't like a person. 

- The people who were in it. 

2. Too much noise and general disruption. , 

- Some people didn't like it and didn't do anything. 

- When you | teacher on the tape! were talking my friends were 
talking too. 

^ - Some people didn't follow directions*, and messed around and 
talked. 

- jNamed personj fooling around. 

- I did not like when everyone started to be bad. -* 

- Sometimes everyone^ fooled arpund. 

• ' it 

" Everybody yelling and shouting. 

- That everyone was not cooperating. 

- I did not like when we were loud. 

- They didn't cooperate. 

- People would fuss over any little thing. 

6 

3. Being exposed to negative situations. 

- When you say your feelings out loud. 

The fags called me names, didn't let me be leader, and said they 
hate me. ' . 

- When the boys teased me. ^ 

- I was yelled at. ' 

- When other people were the leader, they were bossing- me around. 
' - When everyone would talk about my friend. 
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Teacher Reactions 

Teachers reported their reactions on a series of survey forms. 

Teachers ' perceptions of program ,appeal to students * Responses 
given by teachers to an item asking them to tate the four co.iiponents af 
the program — story installments, tape- led games, Heart to Heart talks, 
and optional activities — with respect to student enjoyment are pre- 
sented in Table 13. Teachers could select f roic 5 alternatives: "a 
great deal;" "quite a lot;" "some;" "a little;" and "very little." 
None of the teachers rated any component in the two lowest- categories. 

Data were available for only four sites, C, D, E and On the 

whole, teachers felt that their students enjoyed the course components 

■i. 

either "some" or "quite a lot." The teacher from Site'D reported that, 
in her opinion, her students ' enjoyed the Heart to Heart talks, and the 
optional activities "a great deal." It is interesting to note that", 
comparatively, the teachers from the two inner city^j^tes for which 
data were available (E and F) gave the least' fgivorable ratings to the 
components. All but one of their ratings fell into the "some" category. 
Tape- led Lessons . Teachers were asked to rate the taped lessons 

according to the following six dimensions: (1) volume; (2) spaced (3) 

■ • • / 

appeal of characters; (4) appeal of storyline; (5) conceptual clarity; 
and (6) overall quality. Responses could be selected from five alterna- 
tives: very poor, so-so, good, and very. good, 

*Only 4 teachers tiompleted the questionnaire in which the relevant item 
was found. 
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. Table 13 

Teachers.V-Ratlrigs of Program Compoiten-ts 
With Respect to the Enjoyment of Their Students, 



Program Components 




Degree of Enjoyment 




Story Installments 


E, F 


c, 






Tape-leci Games 


F 

>• 




D, E 




Heart to ^Hea^<;^^^^ 


r E, F 


c 


D 


Optional Activities 


E 




F 


D 



Schools for which data were available: C, D; E, F. 
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Figure 7 presents the teacher reactions to the taped lessons with 



respect to the first two dimensions, volume and pace. On an averaga, 
83.3% of the teachers, with a range of 57% to 100%, rated the volume of 
the taped lessons as being either good or very good. The mean percent 
of teachers rating the lessons as either good or very good with respect 
td^ace was 88.&%, with a range of 71% to 100%. 

Figure 8 presents the results of the analysis of ratings given by 
teachers to the taped lessons with respect to appeal of characters and 
appeal of storyline. The mean percent of teachers rating the lessons 
as either good or very good with respect to appeal of characters was 
87.7, with a range of 66% to 100%. The mean percent of teachers rating 
the lessons as either good or very good with respect to appeal of 
storyline was 89. 7, - with a range of 50% to 100%. 

Figure 9 presents ceacher reactions' to the taped lessons with res- 
pect to conceptual clarity and overall quality. On an a\erage, 85.1% of 
teachers, with a range of 57% to 100%, rated the lessons as being either 
good or very good with respect to conceptual clarity. The mean percent 
of teachers rating the lessons with respect to overall quality as being 
either good or very good was 91.8, with a range of 50% to 100%. 

Heart to Heart talks . Teachers were asked to report whether they 
were enthusiastic or unenthusiastic about the concepts presented ifi 
each Heart to Heart talk. The results of the analysis of the responses 
given by teachers are presented in Figure 10. The^ mean percent of 
teachers having felt enthusiastic about the concepts presented in the. 
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= .volume 



B- = pace/speed 



Figure 7; Percent of teachers rating taped lessons 
with respect to volume and pace/speed as 
either "very good", or "good" shown as a 
function of lessons. 
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Figure 8: Percent of teachers rating the taped lessons 
with respect to appeal of characters and 
appeal of storyline as either "very good" or 
"good" shown as a function of the lessons. 



lOO-a- 



90 



80 - 



y 70 



60 



50 - 




1 2 3 4 5 7 



Q- = 



conceptual 
clarity 

overall 
quality 



"~» — ' — J — I — I — I — I I I — I — I I I I — I — I— I 

9 10 11 13 14 15 16 18 19 20 22 23 24 26 27 28 30 
TAPED LESSONS 



Figure 9: Percent of teachers rating the taped lessons 

with respect to conceptual clarity and overall 
quality as either "very good" or "good" as 
shown as a function, of the lessons. 
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Fieure 10: Percent of teachers reporting to have Figure 11: Percent of teachers having foun the 
. been "enthusiastic" about the concepts talks to be "very worthwhile s own as 

of the Heart to Heart Talks shown^ as a ^ a function of the individual talks, 

function of the individual talks. 
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Heart to Heart talks was 91, with a range of 66% to 100%. Teachers were 
also asked to rate the Talks according to how "worthwhile" they were for 
their students. Teachers could select responses from three alternatives; 
not worthwhile, fairly worthwhile, and very worthwhile. The results of 
the analysis of the responses .given by teachers to this item are pre- 
sented in Figure 11. The mean percent of teachers having felt that the 
Heart to Heart talks were "very worthwhile" was 75, with a range of 33% 
to 100%. Heart to Heart talks y/12 and //25 received the highest ratings 
in terms of both enthusiasm and value. Teachers questioned the value 
of the last Heart to Heart talk. 

Optional Activities . The frequency of optional activity use during 
the field test by each site is presented in Table 14. The figures re- 
ported represent the number of times teachers reported using activities 
following the taped lessons, and should not be misinterpreted as repre- 
senting the number of actual activities used by individual . teachers 
after lessons or at other times. Data on the actual number of activities 
used were available for only three sites (B, C, andG) and have aTso 
been reported in Table 14.* 

For Unit I, teachers could have used optional activities after each 
of 10 taped lessons. None of the teachers reported using optional 
activities after every lesson (100% of the time), but the teacher from 
site B used activities after 9 lessons (90%), teachers from sites A, F, 

*The report form which required teachers to fill in the name of all 
activities used for a certain lesson was not submitted by the other 
sites. 
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Table 14 



Frequency of Optional Activity Use 
For Each Site by Unit " 



1 1 

SITES - 


UNIT 


COURSE 


TOTAL NUMBER 
OF ACTIVITIES 
USED 


I 


II 


III 






A 


*8 

30 


3 

.--^^^ 42 


3 

^>^8 


14 


+ 


B 


9 


7 

LOO 


7 

^'^i^o 


23 


63 


C 


40 


3 


2 

^^28 


9 


15 


D 


^''^70^ 


3 

^'^^ 42 


5 

^"-^ '1 


15 
^^62 


+ 


E 


5 

^-^50 


0 

.^^^ 0 


0 

0 


5 

^^^20 


+ 


F 


8 


3 

42 


0 

.^-^ 0 


11 

^^^45 


+ 


G 


8 


6 

^'''''^ 85 


4 


18 


32 


Unit 
j Frequencies 


,49^^^ 
70 


25 

51 


21 ^.x^ 


95 
^^i6 





+ = No data were available for these sites. 



* The figures to the left of the diagonal represent 

the number of times teachers used optional activities*, 
The figures to the right of the slash are percentages 
representing the number of times activities were used 
divided by the number of possible times ' 



and G used activities after 8 lessons (80%), and the teacher from site 
D used activities after 7 lessons (70%), Teachers from sites C and E 
reported using activities the least number of times, 4 (40%) and 5 (50%) 
times, respectively. Looking at Unit I as a whole, the maximum number 
of total times activities could have been used after lessons was 70 (7 
sites multiplied by 10 lessons). Teachers, for Unit I, reported using 
activities 49 out of the'possible 70 times, or rather, 70% of total 
times. 

For Unit II, teachers could have used activities after each of 
the 7 taped lessons. Only one teacher (B) reported using activities 
the maximum number of times; sites G, F, D, C, A, and E used activities 
6 (85%), 3 (42%), '3 (42%) , 3 (42%), 3 (42%), and 0 (0%) times, res- 
pectively. Out of a possible maximum of 49, for the unit as a whole, 
teachers reported using activities 21 times, or 42% of the possible 
total. 

For Unit III, teachers, again, could have used activities after 
each of 7 taped lessons. The teacher. from site B reported using acti- 
vities the maximum number of times, with sites D, G, A, C, E, and F using 
activities 5 (71%), 4 (57%), 3 (42%), 2 (28%), 0 (0%), and 0 (0%) times, 
respectively. Looking at the unit as a whole, teachers used activities 
21 out of the 49 maximum total number of times possible (or, after 42% 
of the lessons). 

The data available on the number .of activities used are reported in 
the right hand column of Table 14, The teacher from site B used a total 
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of 63 activities and the teachers from sites C and G used a total 
of^32 and 15 activities, respectively. Of the 63 activities used by 
the teacher from site B, most were of a type involving discussion, and 
drawing and/or writing. The teacher from site ,C primarily use4 the 
ditto sheet activities provided in the manual. Most of the activities , 
used by the teacher from site G were of a -'paper and pencil" naturej. 

l^en asked to indicate their reasons for using an optional activity, 
teachers consistently noted all four of th^ offered response alterna- 
tives: (1) I liked it; (2) I thought students would like it; (3) it 
reinforced lesson objectives; and (4) it reinforced course objectives. 
Other reasons for using an activity that teachers noted were: (1) it 
motivated the students; (2) it expanded language skills; (3) it increased 
creativity; and (4) it clarified concepts. The almost unanimous reason 
given for having decided not to use an optional activity was lack of 
time. The teacher from- site E also reported that her students were too 
uncooperative, especially near the end of the program, to be able to 
complete an activity. ^ 

Teachers reported having found the optional activities to be en- 
joyable to their students (see Table 14) and easy to administer (see 
Table 15). During their interviews, they commented that the activities 
were very helpful and, in fact, necessary for clarifying and reinforc- 
ing lesson and program concepts. The most frequent suggestion made was 
that the activities should involve the students more actively, to con- 
trast with the sitting, listening and coloring required for the taped 
lessons^ 
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Importance/Relevance of Program ^ Determining how important or 
relevant the program is with respect to the students' personal develop- 
ment and to education in general was considered an important aspect of 
an evaluation pertaining to appeal and acceptability. Because of the 
age- of the students, information relating to importance and relevance 
was solicited from teachers. 

Teachers were asked to rate each taped lesson with respect to. how 
important they fel-t the contents were to their students' lives on a five, 
point scale: not important; a little important; fairly important; very 
important; and extremely important. The results of the analysis of 
these responses are presented in Figure 12. On an average 72*9^ of the ' 
teachers, with a range of 60%, to 100%, found the contents of the taped 
lessons to be very important or extremely important to their students' 
lives. • ' 

Teachers were also asked to write in responses to questions con- 
cerning the importance of the program goals to elementary education and 
to their students* personal development. Only four of the seven teachers 
participating in the program (B, D, E, F) responded to the questionnaire 
in which these two questions were found. The responses given by these 
four teachers are 'presented below. 

How important are the course goals to elementary education? 
*«» . 

(B) "Very important and relevant to 4th grade 
peer problems." 

(D) "in order for a .child to reach full poten- 
tial academically, he must be aware of 
himself and the world about him and be 
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Figure 12: Percent of teachers finding the contents of 

the taped lessons to be "extremely important" 
or "very important" to their students' lives 
shown as a function of lessons. 
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able to understand and cope with feelings 
and problems that arise. He must develop a 
good self concept and feel accepted by 
peers^ as well as the adult world. Heart- 
smart takes this all into account and works 
it out beautifully for the children. 'Hie 
program guides them into a great deal ot 
'soul-searching and understanding of human 
behaviors* 

(E) "Extremely/children must learn to be less 
selfish." 

' ' (F) "Good." 

* • \ 

How important are the course goals to students' personal development? 

(B) "Very important. I think the children ' 
will remember a lot of whart^they learned." 

(D) Answer included in above statement. 

(E) "Very important — (children) must learn 

to share and recognize others' feelings." 

(F) "Very, helps to direct your thinking." 

Ease of Administration . Teacher reactions toythe program with 
respect to ease of administratibn were, considered important to deter- 



mining the overall acceptability of the program^ es^^ecially for educa- 

1 V ' . ■ 



tors. 



Teachers were asked to rate the four components of the program — - 
story installments, tape-led games. Heart to Heart\ talks, and optional ' 
activities — according to how easy o'v difficult they were to administer. 



Responses could be selected from four alternatives: 



very difficult. 



difficult, easy, and very easy. The ratings given by the teachers are 



presented in. Table 15. Data were available for only 
E, and.F. All four of the teachers responding found 



four sites: B, D, 
the story 
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installments, the taped games, and the optional activities easy or very 

easy to administer. Three of the four teachers found the Heart to 

«, - ■ - 

Heart talks easy or very easy to administer and one teacher found the 
talks difficult to administer, 

* Table 15 \ ' ^ " 

Teachers' Ratings of Program Components . 
With Respect to Ease of Administration, 



Program Components 




Ease of Administration 






Story Installments 






E 


B, D, F 


Taped Games 






B, E 


^ D, F 


Heart to Heart Talks \ 




F 


B, E 


D 


Optional Activities 






b', E,. F 


D 



Schools for which data were available: 
B, D, E, F 
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Responses given by- these sane teachers to other items relating to 
administration also indicate th;at the program was considered easyxto - 
implement. All four teachers found ^ the program materials easy or very- 
easy to prepare, collect,. and store, and all reported having encounter- 
■ ed no procedural- difficulties when administering the program. Two of 
the four teachers felt that the program required no more preparation 
time .than other subjects; one commented that in order to have a good 
_ lesson, the materials needed to' be thoroughly checked and that this 
required more ,p reparation time than usual; and one offered no 



response. 




V. STUDENT ACHIEVEMENT 



.Student achievement refers to the performance of students on the 
Game- and Review^ages which followed and wert, completed after the pre- 
sentation* of the. "taped story. Fx)r evaluation purposes, a random -sample 
of fi^e s^ts__of„_s.tuderit ^materials -was selected -f rom-each-site- -f or each- 
lesson. (Emphasis has been placed on the Reviewpages, since these, more 
so than the Gamepages, were- considered to be indicators of student com- 
prehension of individual lesson concepts.) ^' 

Figure 13 presents the results of the analysis of answers given by 
students to the questions on the Reviewpages. On an average, only 4.4% 
of all the. answers given for all the review questions were inporrect 
^ (with a range of ,6% to 9.7%). Lesson 27 had the most number of.in^ 
correct responses given to its review questions. Lesson 28 ha4 the 
least number of incorrect responses given. . • 
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Figure 13: Percent incorrect answers 
given by students to- 
review questions.* 
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*Taped lessons' #13 and' #30 
had no r/view pages. 
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Students were conscientious in completing the Gamepages, and for, 
the most part were able to follow the instructions given by the tape 
and complete assigned tasks satisfactorily. The few problems that were 
apparent were due primarily to confusing page layouts and insufficient 
clarification of the taped directions. 

VI. PROGRAM EFFECTIVENESS 

Preceding sections of this chapter dealt primarily with formative 
evaluation results with respect to program antecedents, program trans- 
actions* and, to some extent, program outcomes. However, the focus of 
this section is on program outcomes as related to the goals/objectives 
of the program. The first part of this section takes the approach of a " 
congruency analysis in which the question of how successful the program 
was in achieving its stated goals/objectives is addressed. Four hypo- 
theses are stated and results which provide evidence to support or re- 
ject these hypotheses are reported. Comparing differences between 
experimental and comparison groups on measures of program effectiveness, 
and reporting results from interviews conducted with experimental stu- 
dents and teachers establish the bases for the congruency analysis. 
The second part of this section considers two antecedent factors, pupil 
sex and classroom environment, and several program implementation fac- 
tors in relation to program outcomes in the manner of a contingency 
analysis. The questions addressed in this part tend to be exploratory 
and suppositional in nature father than based upon a specific a priori 
set of goal-based evaluation h^otheges. The overall objective of this 

1 
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section is to report results bearing on the relationships among treat- 
ment condition, contingency factors, and program outcomes. 
Congruency Analysis 



Hypothesis 1 


The Heartsmart program students will demonstrate greater ability 


and/or 


tendency than comparison group students to: 


• 


Identify feelings and behaviors, and infer feelings from 
behaviors in others; 


• 


Identify ways in which people may behave when they have 
certain feelings; ^ * 


• 


Identify the cause and effect relationship between feelings 
and behavior; 


• 


Determine feelings in others by asking; 


■ • 


Express individual needs and desires and take (appropriate) 
actions to accomplish them; and 


• 


Express feelings. 



The multidimensional hypothesis reflects the, interest of the 
Heartsmart program to improve interpersonal skills primarily in attri- 
butes related to the domain of "feelings." Each dimension corresponds 
to a specific program objective. The hypothesis was tested by examin-j 
ing results from the two criterion-referenced tests used in the field 
test, V?hat's Happening? and V?hat Would You Do? in terms of differences 
between experimental and comparison classes. 

Table 16 displays MANOVA statistical results of each field test 
site on the above two measures. Thr important parts of the table to 
inspect are the e>:perimental and comparison group mean scores for each 
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Table 16 ' 
Multivariate and Univariate ANOVA Results 
of Experimental vs. Comparison Group Differences 
on the What's Happening? and What Would You Do? Tests, 
Reported by Each School 







Experimental 


Comparison 




SCHOOL 






N 


MEAN 


N 


MEAN?, 


UNIVARIATE 
F-TEST 


A ' 


What's Happening? 
What Would You Do? 




2,3 
23 


16.22 
9.82 


21 
21 


12.90 

A A 9 


6.84** 
12.10** 




MANOVA 


?(2.39) = ^•^S** 






B 


What's Happening? 
What Would You Do? 




28 
28 


13.43 
11.36 


28 
• 28 


13.21 

/ • O U 


0.05 
12.36** 




MANOVA 


F V = 6.12** 
(2,51) • 






C 


What's Happening? 
What Would You Do? 




28 
28 


14.42 
6.67 


23 
23 


12.04 


6.01** 
2.01 




MANOVA 


^2.46) = 






D 


What'^s Happening? 
What Would You Do? 




24 
24 


15.33 
12.88 


24 
24 


14.63 

9 62 


-0.68 
^ 7.86** 




MANOVA 


^2.43) =^-07* 






E 


What's Happening? 
What Would You Do? 




„ 28 
28 


12.71 
9.42 


24 
24 


10.87 


4.15* 




MANOVA 


F = 5.77** 
*(2,47) 






F 


What's Happening? 
What Would You Do? 




30 
30 


14.03 
9.20 


31 
31 


10.32 
7.38 


21.66** 
4; 57* 




MANOVA 


^2.56) = 11-^6** 






G 


What's Happening? 
What Would You Do? 




23 
24 


12.43 
8.54 


2S 
23 


11,04 
6.69 


1.53 
7.74** 




MANOVA 


7(2.41) =3.99* 







* slgrilf leant at the p<.05 level 
** significant at the p<.01 level 



measure and the multivariate and univariate F-test results. The 
MANOVA F values, signifying the results of testing differences between 
two centroids between experimental and comparison groups on both mea- 
sures, are statistically significant (p<.05) at all field test sites. 
Treatment differences are statistically significant at the p< .01 level 
in four schools and at the p<.05 level in the remaining three schools. 
The univariate F-test results show that experimental classes score sig- 
nificantly higher on the average than comparison' classes on either or 

both tests in all seven schools. At school A the mean score differences 

o 

on each test are highly significant (p<.01). The results at schools 
B, D, E, and G indicate very strong differences (p<.01) between groups 
on the What Would You Do? test; and at school F, group differences 
turned out to be significant at the p^.05 level. In schools C and F, 
univariate tests indicate highly significant differences (p<.01) on 
the What's Happening? test. All univariate test results differentiate 
experimental and comparison groups consistently in favor of experimental 
groups. The univariate results found in school E may be questioned on 
the grounds that the assumption of homogeneity of variances between 
groups was not met (Hartley's F-max^^ = 2.40, p<.05). The largest 

variance was found in the experimental class; it is also the class 
which contains approximately one- third fifth grade pupils. 

It is instructive to examine Hypothesis 1 in terms of its separate 
dimensions. Table 17 presents the mean percents of experimental and 
comparison group subjects giving correct answers to seven clusters of 
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Table 17'' 

Mean Percentage of Experimental and Comparison Group Subjects 
Giving Correct Responses to Selected Item Clusters on 
the What's Happening? and What Would You Do? Tests 



<3 

Program Objective 


Items-Item Cluster 
Test 




Mean Percentage (Z) of Subjects Giving Correct 
Responses to Item Cluster (Program Objective) 


2 -Test 


What's 
Happening? 


What Would 
You" Do? 


0 

u 
u 


School 


Average 
Total 


A 
A 


11 
i) 


c 


y 


V 
L 


r 


(I 


1, Identify (infer) feelings 
and behaviors and infer 
feelings from behavior in 
others. 


Al, Bl, 
B4, CI, 
D2 HI 

U*t 1 111 




E 

c 


71.7 

53.6 


57.0 

53 6 


51.6 

56 0 


66.4 

65 1 


70,2 

44 2 


54.5 
51 7 


56.8 

47 q 


61.2 
53.1 


2.24* 


2. Identify ways in which 
people may behave when 
they have certain feelings 


C2, D3, 
£2"^, Gl 




E 
C 


68.2 
52.8 


54,5 
41.1 


71.1 
52.9 


56.5 

53.2 


49,1 
42.1 


62.4 
36.7 


52.9 
39.2 


59.2 

A5 L 


3.81* 


3.1 Identify cause and effect 
relationship between feel^ 
ings and behavior, . 


A2,B3,C3, 
D1,F1,F2, 

H2 . 




'E 
C 


73.3 
54.8 


58.4 
64.0 


64.7 
50.9 


61.0 
67.1 


58,5 
54.9 


61,9 
54.2 


53.0 
51.4 


6U5' 
Sfi 7 


1.33 


4« Determine feelings in 
others by asking (not 
by guessing). 


B2, C4, 
Ela» 

Pir 


2, 7, 10 


E 
C 


66.1 
70.6 


65.5 
64.5 


55.2 
49.1 


75.3 
62.5 


60.4 
42.2 


62.7 
47.5 


60.7 
43.7 


63.6 


2.60** 


^. Express needs and desires 
and take actions to ac- 
complish thenii 




3, 4, 6, 
8, 14, 16, 
17, 18 


£ 

C 


60.9 
36.3 


69.6 
40.8 


38.0 
38.9 


77.0 
59.8, 


53.3 
37.6 


51.5 
39.3 


50.4 
39.0, 


57.2 
41.6 ■ 


4.32« 


6. Express feelings. 




I, 5, 9, 

II, 12, 
13, 15 


E 
C 


39.1 
28.5 


48.2 
36.8 


29.6 
40.8 


54.8 
37.9 


38.6 
34.5 


40.5 
36.8 


39.1 
27.0 


41.4 
34.6 


1.92 


TOTAL 


Experimental Group 
Comparison Group 


57.3 
47.6 . 


2.67*1 



. * P^.05 analyzed as percentage of subjects giving at least one correct response. 

** p<.01 



items, each corresponding presumably to one of the dimensions (i.e., pro- 
gram oBjectives). Item clusters were formed on a logical basis by con- • 
tent analysis, Z-tests were performed to test for differences between • 
proportions of correct responses given by experimental and comparison 
pupils for each item cluster. The results indicate that a significantly 
higher percent (p<,025, one-tailed test) of experimental subjects across 
all seven schools gave correct responses to four of seven item clusters 
than did comparison group subjects. Experimental students performed 
better on Objectives 3 and 6, but differences were not statistically 
significant. When, all program objectives were considered in a single 
overall analysis, a significantly higher percent of experimental students 
gave correct responses to the tests' item clusters than the comparison 
students (p < ,01) . 

• The results from the above two sets of statistical analyses appear 
to provide substantial support for Hypothesis 1. For the most part, the 
results are both highly significant in a statistical sense and consistent 
across field test sites. To the degree that the two measures used in 
the analyses can be taken as valid measures of program object-LV^.s, 
Hypothesis 1 can be judged as havingbeen strongly supported by the 
evidence. However, since the test of Hypothesis 1 was perfornzed on 
posttest measurements only, the possibility of pre-existing group 
differences in favor of experimental classes cannot be completely ruled 
out as an alternative explanation for the obtained results. 
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Hypothesis lA 

Heartsmart program students will report they have learned interl- 
opers onal skills consonant with the objectives of the program. 



Hypothesis 1 was stated as a comparative prediction that experimen- 
tal classes would score higher than comparison group classes on program 
effectiveness assessment measures. Hypothesis lA is a corollary to the 
first hypothesis in that it relates to the identical substance (i.e., 
interpersonal skills in the "feelings" domain), but it does not make a 
comparative prediction. Data gathered at the end of the course by 
interviews with approximately a 20% sub-sample (N = 38) of experimental 
subjects, presumably representative of all Hearts/nart-instructed stu- 
dents in the field test, provided valuable information about students* 
perceptions of what they learned from the program. The above hypothesis 
was tested by analyzing the responses to Question 1 of the student inter- 
view according to the same "objectives" categories used in the statement 
of Hypothesis 1* The question was: "Think about Jack and his adven- 
tures, and the things he learned. What did you learn from HeartsmartV^ 
The question is a very open-ended one; a legitimate evaluation concern 



might be raised about the validity of studeat interview results in that 
children may have learned inte^. personal skills taught by the, program 
but could not articulate them in a frt<^- response manner. One might 
assert that the open-ended question method used in this instance pro- 
bably provides a rather stern test of the hypothesis. 

Table 18 shows the results obtained from analyzing student re- 
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Table 18 

Analysis of Student Responses to the Question: 
"What did you learn from Heartsmart?" 



Program Objective 


Number of 
ResDonse*; 


Percent of Total 

NllTTInPT* f Pocnrtncae 

iiuuiucjL ui rvcbpoiibcS 


Percent of Students 
vjxv j.ug Jt^coponoc 


1. Identify feelings and behaviors, and 
infer feelings from behaviors in 
others. 


9 


19.5 


28.1 


2, Identify v^ays in which people may 

behave when they have certain feelings. 


1 

i. 


L . L 


3.1 


3. Identify the cause and effect 
relationship between feelings and 
behavior. 




f. 0 

4.3 


6.3 


4, Determine feelings in others by 
asking. 


5 


10.9 


15.6 


5. Express individual needs and desires 
and take (appropriate) actions to 
accomplish them. 


0 


0.0 


.0 


6. Express feelings. 


. 16 


34.8 


•50.0 


7. Other (i.e., non-classifiable re- 
sponses in above 6 categories) 


13 


28.3 ■ 




100.0 


TOTAL 


46 


100.0 



NOTE: Above results are. based on usable data from 32 experimental subjects. 
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responses to Question 1 of the interview ("What did you learn from Heart- 
smartV). Of 38 subjects interviewed, 32 gave interpretable responses. 
A total of 46 responses (i.e., different ideas /thoughts) were classifiable 
into 6 "objectives" categories, and one "other" category. The "other" 
category contained responses that were relevant to program objectives 
outside of 'the "feelings" domain being examined here. The second column 
of numbers indicates the percent of all responses that were classified 
in each of the seven categories. The final (right hand) column shows 
the percent of' students giving responses classified into the 6 "objec- 
tives" categories. Attention is focused primarily on this set of results. 
Fifty percent of the students responded with objective 6: expressing 
feelings; students reported learning that it was "good" or "O.K." to tell 
and/or show their feelings, to share feelings with others, and to express 
negative feelings (e.g., anger). Objective 1, identify feelings and 
behavior and infer feelings from behaviors in others, was expressed by ^ 
more than one-fourth of the students (i.e., 28.1%). Objective 4, 'deter- 
mine feelings in others by asking, was expressed by 15.6% of the students. 
Very few students responded in ways relating to objectives 2 and 3, and 
none of the students expressed objective 5. 

These findings would appear to support Hypothesis lA to the extent 
that most students interviewed (32 out of 38) reported in a free-response 
manner learning interpersonal skills from Heartsmart Adventures that 
were consonant with 5 of the 6 program objectives in the "feelings" 
/ • ■ . ' - 
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domain. The evidence to support this hypothesis must be qualified in 



Hypothesis 2 

Heartsmart program students will transfer interpersonal skills 
learned from the program to ot her situations. 

An objective stated for the program was to increase the likelihood 
that pupils would apply interpersonal skills they learned from the 
program to situations outside the classroom. This hypothesis was exam- 
ined by an analysis of pupils' rersponses to Questions 2, 3 and 5 of the 
student interview. The results are. reported below. 

Of 38 interviewees, 33 reported that they show their feelings more 
now than they did before taking the program. Five subjects said that 
they do not show their feelings more now than before. Three of these 
five ^students reported they shew their feelings Less, now than they did 
— before. When asked what kind of feelings' they show now, 17 subjects 
responded with some combination of the feelings taught in the course 
(e.g., mad, sad, gl^d, hurt, happy). Ten responded with a catch-all 
phrase "all feelings," and six subjects described situations in which 
they expressed or acted out feqlings rather than naming the feelings. 
Wtien asked where, or in what situations, they show their feelings now, 
nine subjects reported ones occurring mostly at home (with parents and. 
siblings), 7 responded with "everywhere," six students reported showing 



this respect: a very small number of .subjects reported having 



learned interpersonal skills related to three program objectives, 2, 3 . 



and 5. 



1?0 




feelings more only with their. friends , one subject answered Vjust at 
school," and ten reported showing feelings at home and at school. 

The interview asked students to relate a slto^tion in which they 
expressed their feelings. Thirty-one subjects 'resporTd^ to this ques- 
tion by portraying a variety of situations in which they expressed a 
variety of feelings. Sixteen students reported situations involved 
with siblings and friends, and 14 of these 16 ware situations in which 
siblings /friends were told to stop an annoying behavior. " Seven stu- 
dents related situations in which feelings were expressed to parents, 
such as mad, sad, and scared feelings. Four students related situations 
in which they expressed feelings to adults outside the classroom. Four 
students told of situations in. which they wanted to express their feel- 
ings, but didn't. 

Earlier it was reported that three students said tihey showed their 
feelings less often now than before the program. One of these could not 
give a reason why, and the other two se med to show an understanding of 
when it was best (for them) to hide feelings (e.g., when telling feel- 
ings might cause others .to have hurt feelings, and when expressing ceel- 
ings might bring on further trouble for themselves). 

Question 5 as^ed pupils if showing feelings, helped them^in their 
interpersonal relationships. Thirty subjects answered "yes," 2 answered 
"no," and 5 said "sometimes." The 35 "positive" respondents were asked 
to explain how showing feelings helped them; 17 subjects stated that 
expressing their feelings "made them feel better," and 15 students said 
that some desired end was accomplished (e.g., getting a friend to stop 
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an annoying behavior, getting friends to be nicer to them). The re- 
maining three subj-ects did not give interpretable responses. 

Also worth. noting with respect to Hypothesis 2 are several res- . 
ponses to "Questions 14 and 17 of the Teacher Interview indicating that 
students had become more confiding, and that they applied interpersonal 
skills outside the classroom. 

In summary, datk gathered from experimental subjects by^^lnterview . 
at the end of the program provide support for Hypothesis 2. Since the 

1 

interviews primarily dealt with only dne program objective, "expressing- 

4 feelings," It is difficult to assess how much transfer of learning from 

' i " 

the program occurred in other objective5^ Data supporting Hypothesis 2 
are directed to a relatively narrow range of ^program objectives; it 
cannot be definitively stated, therefore, that interpersonal skills re- 
lated to other program objectives have been applied in out-of-class 
interpersonal situations. . ^ ^ 



Hypothesis 3 < / - 
The social climate of Heartsmart program classrooms will become 
increasingly more cooperative and satisfying than that of comparison 
group'' classrooms. ^_ o 



Th^s hypothesis was tested by comparing experimental vs. comparison 
group classes on three scales of the My class 'Inventory (MCI), using a 



repeated measures design MANOVA technique. Student and teacher inter- - 
views also provided relevant data for examining this hypothesis. The 
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results from the MCI are repo^rted first. 
My Class Inventory 

The goal stated for the program was that students will be able to 
cooperate more effectively with others by the end of the program. It is 
necessary to point out that the MCI scales are not measures of coopera- 
tive behavior, but rather measures of the level" of cooperation which stu- 
dents perceive (or report) in their classroom. As used here, the con- 
cept of cooperation is defined in perceptual rather than behavioral 
terms-.and is a characteristic of the environment and not individuals. 
The test of Hypothesis 3 is based on the reasoning that if students in 
Heartswart classes exhibited more, cooperative behavior as a result of \ 
the program, then one would expect students to characterize the social 
climate of the classroom in terms of greater cohesion, friendliness, 
decreased hostility, and the like. The meaning of cooperation is best 
captured by the Friction(F) and Cohesion (C) scales of the MCI. These 
scales are negatively correlated with each other. The Satisfaction (S) 
' Scale is negatively correlated with the Friction (F) Scale and positively 
correlated with the Cohesion (C) Scale. These relationships suggest 
that the S 'Scale may also be considered as an index of cooperation. 
Hypothesis 3 thus translates into the prediction that the mean scores 
from pre- to posttest should increase on the S and C Scales and decrease 
^ on the F Scale for experimental classes, and that the pre- post mean 
differences should be significantly larger for experimental than for 

i 

comparison classes. 
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Among experimental classes, only the one at school D showed Lliis 
pre- to posttest pattern of change, i e. , an increase ia Satisfaction 
and Cohesion^'^vand a decrease- iii Frict 'on. 

Table Iw^presents MA?'OVA and univariate F-test results comparing 
^experimental/ vs . comparison group classes in each school on -^an pre- to 
posttest change scores for threa MCI Scales: S, F, and C, 

The results revealed that multivariate F's were significr t at two 
of the seven schools: school B at the p^ ,05 level, and school E at the 
p4»01 level. At school B the Friction Scale discriminated most between 
experimental and comparison classes; in school E, Satisfaction and Co- 
hesion Scales exhibited the most discriminating power. Contrary to 
expectation, at school B the mean score on the F Scale increased in the 
experimental class, decreased in the comparison class, and the difference 
between the mean changes between experimental and comparison classes 
(interaction effect) was highly significant (p<.01). Only one school, 
C, with non-significant multivariate F, showed significant univariate 
F-xatios. At school C, the Cohesion Scale was involved with the mean 
score decreasing in the experimental class for pre- to posttest and in- 
creasing in the con:parison class. 

Student and Teacher Interviews 

\' 

\ Question -on the student interview form, and its sub-questions 

were analyzed to assess student perceptions of the classroom' environment. 

\ 

When asked if the children in their class had changed since taking 
Heartsmart, 20 subjects out of 38 answered "yes," 10 responatd with a 
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Table 19" 

Multivariate and Univariate ANOVA Results on the 
Satisfaction, Friction, and Cohesion Scales 
of the My Class Inventory 



SCHOOL 


SCALE 


Experimental Comparison 


UNIVARIATE 
- F-TEST 






N PRE POST CHANGE 


N PRE POST CHANGE 


A 


Satisfaction 

Friction 

Cohesion 


23 7.23 4.92 -2.30 
23 5.84 6.63 +0.79 
23 6.03 5.39 -0.64 


21 5.39 4.00 -1.39 
21 6.51 7.91 +1.40 
'21 5.82 5.13 -0.69 


2.57 
0,94 
,00 




MANOVA F,_ « 1.99 


B 


Satisfaction 

Friction 

Cohesion 


28 5.96 4.82 -1.14 
28 5.89 6.53 +0.64 
28 6.28 6.10 -0.18 . 


28 5.99 4.74 -1.24 
28 5.85 4.99 -0.85 
28 5.25 5.53 +0.28 


0.02 
. 8.16** 
1.02 




. ™* ^3.50) " 


C 


Satisfaction 

Friction 

Cohesion 


28 5.64 3.71 -1.85 
23 6.50 7.07 +0.57 
28 5.64 5.07 -0.57 


23 3.65 2.04 -1.61 
23 7-19 7.34 -0.05 
23 5.30 5.78 0.48 


0.12 
1.31 
5.57* 




MANOVA F., .cv - 1.85 
(3,45) s 


D 


Satisfaction 

Friction 

Cohesion 


24 5.69 6.52 +0.33 
24 6.78 5.30 -1.47 
24 5.82 6.57 +0.75 


24 6.16 6.25 +0,09 
24 5.25 5.16 -0.09 
24 6.10 5.91 -0.19 


1.28 
3.50 
3.03 




MANOVA F,- - 1.80 < 


E 


Satisfaction 

Friction 

Cohesion 


28 4.06 3.60 -0.46 
28 7.24 8.03 +0.79 
28 5.24 4.9^9 -0.24. 


25 5.56 3.12 -2.44 
25 5.94 7.30 +1.36 
25 6.98 5.52 -1.45 


10.14** 
1.28 
5,02* 




\ MANOVA F.- - 4.85** 
\ (3,47) 


F 


Satisfaction 

Friction 

Cohesion 


31 4.19 \3.48 -0.71 
31 7.58 &,.19 +0.61 
31 5.22 4>^ -0.39 


31 3.87 3.05 -0.82 
31 5.73 6.31 +0.58 
31 5.74 5.83 +0.09 


0.02 
0.00 
1.13 




N^"^* ^3.56)."°-*« 


G 


Satisfaction 

Friction 

Cohesion 


24 4.70 3.95 -0^5 

24 6.62 7.41 +0.79\ • 

24 5.79 6.08 +0.29 \ 


23 5.99 5.43 -0.56 
23 6.93 7.05 +0.12 
23 6.87 6.55 -0.32 


0.09 
1.29 
1.16 




MANOVA F,- ,\ - 0.32 
(3,41)^^ 



* olgnlf leant at the p<.05 levei^ 
** significant at the p<.01 level.^ 
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qualified "yes," (e.g., "yes, some"), and 8 said "no." Students were 
asked to give an example of how the class had clianged. Thirteen sub- 
jects stated that children in the class were "easier to get along with." 
Ten students responded that children were "friendlier and nicer" and 
that they talked their difficulties out more; 2 said that class members 
were more understanding of each other; 2 said class members had. more 
feelings now. Three subjects (from the same school) reported that the 
class was different because the children fought more. 

When asked if they "liked" the class better now than before Heart- 
smarty 22 subjects reported the class was better now , 3 said it was 
better before , 2 said neither , 3 said it was the same , and 8 gave no 
answer. In giving examples of how the class was better now, students 
unanimously reported with such answers as: the children were easier 
to get along with; were kinder, nicer, more friendly and helpful, and 
more caring of others; and were more willing to do more sharing with 

-J ^■ 

each other. 

Interviews conducted with teachers were not specifically directed 
to questions about changes in the classroom environment. Nevertheless, 
to open-ended questions concerning changes in student behavior (Question 
14) and benefits accrued from the program (Question 17), at least three 
teachers reported that they thought students had become more cooperative 
and accepting of each other. 

In summary, the results obtained from the MCI and student and 
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teacher interviews give partial support to the hypothesis that the 
social climate of Heartsmart program classrooms should become more 
cooperative and satisfying by the end of the program, and that compared 
with non-program instructed classrooms, sifch changes would occur to a 
greater degree. Multivariate results from the S, F and C Scales of the 
MCI indicate that at five of the seven schools in the field test no 
significant treatment group differences were found. At one of the two 
schools which did show significant group differences on the multivariate 
test, the hypothesis prediction was supported; at the remaining school, 
in the Heartsmart program class, scores declined on the S and^ C Scales 
and increased on the F Scale from pre- to posttest, but in th^ compari- 
son group, this same trend was even more dramatic. \ 

The student and teacher interview data, on the other hand, do not 
always support the findings obtained on the MCI. Both students and 
teachers reported higher levels of cooperative behavior and less fric- 
tion among students. These somewhat inconsistent results are not easily 
reconcilable. Chapter Five of this evaluation report will delve further 
into an explanation of or speculation about these results. 
FIRO-BC 

No hypothesis is stated that relates to the FIRO-BC. In Section V 
of Chapter Three, the measure^ is briefly described in terms of content 
and purpose; Section VII of the same chapter discusses perceived limita- 
tions and difficulties related to the use of this instrument as a mea- 
sure of program effectiveness. In view of these considerations it was 
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decided that it might well be presumptuous to generate hypotheses; the 
alternative of presenting a discussion of results seemed more appro- 
priate. 

Before presenting the results, it is necessary to define the 
terms used in the FIRO theory, and to describe the scales and their re- 
lationships. Three affective areas are measured, each on two dimen- 
sions. The areas are described below. 

• Inclusion: the degree of relationship with others with respect to 
interaction and association. 

o Control: the degree of relationship with others with respect to con- 
trol and power. / 

• Affection: the degree of relationship with others with /respect to 
love and affection. 

The two dimensions are: / 

• Expressed — the way in which the person perceives himself or 
herself as behaving toward others; 

• Wanted — the way in which the person wants others to behave toward 
him or her. 

An interpretation of scores on the Expressed (E) and Wanted (W) 
dimensions in relation to each of the areas of Inclusion, Control, and 
Affection,' based on Schutz (1966), is summarized in Table 20. 

The scores on each of the six FIRO-BC scales (each of the 3 areas 
X the 2 dimensions) range from 0-9. Thus, mid-point scores fall at 4 
or 5; very low scores range from 0-2; and very high scores range from 
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Table 20 



Interpretation of High and Low Scores 
on the FIRO-BC 



AREA 


SCALE 


DIMENSION 






Expressed 


Wanted 


Inclusion 


High 


I initiate interaction 
with others. 


I want to be included, 
paid attention to by 
others. 




Low 


I exclude others from 
my activities. 


I want others ^o leave 
me alone. 


Control 


High 
Low 


J. lUdlvc UCC -Lo XUliO y J 

to control others. 

I am a follower, tend , 
to submit to others. 


T 'tJ^^n^ ^^VlPrcI tc\ control 
me. 

I want others to submit 
to my authority^ 


Affection 


High 
Low 


I act close and 
personal toward 
others. 

I act cold and dis- 
tant toward others. 


I want others to be 
close , and personal 
toward me. 

I want others to remain 
at an emotional distance 
from me. 



< 
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7-9. Since the FIRO-BC is not an achievement test, no positive or 
negative connotation is attached to any score on one; dimension; a high 
score on a single dimension is no better or worse than a low score. 

As may be seen from an examination of Table 20, high scores on both 
dimensions in the areas of Inclusion and/or Affection are complementary, 
as are low scores on both dimensions for those two areas. When E and 
W scores are complementary, or close, (for Inclusion and Affection) an 
individual is likely to feel comfortable as a person and in relation- 
ship to others. Conversely, when an E score is higher or lower than a 
W score, in either of the two areas, an individual is likely to be dis- 
satisfied as a person and frustra-ted in relationships with others. Thus, 
, . tljjB relationship between dimensional scores for each of these two areas 
is highly relevant. * 

The area of Control is unlike the other two. "There is no neces- 
sary relation between an individual's behavior toward controlling others 
and his behavior toward being controlled," (Schutz, 1966, p. 23). The 
' illustrative example given by Schutz (1966) cites an army sergeant who' 
willingly wields authority over privates, and also willingly submits to 
the authority of officers. Thus, according to • circumstances and situa- 
tions, an. individual could be equally comfortable with complementary ^r 
with opposing scores. 

As stated in Chapter Three, although Heartsmart is structured on 
the FIRO theory, the program is not necessarily designed to change an 
individual's expressed or wanted behaviors in any\of the three areas; 
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rather it helps the individual (somewhat covertly) .to identify and de- 
fine intrapersonal "and interpersonal needs and to express the feelings ' 
on which those needs are based in order to accomplish that which the 
individual wants from others. Thus, if the program accomplished this 
objective, it is just possible, for instance, that an individual scor- 
ing low E Inclusion and high W Inclusion before the program would in- 
crease the E Inclusion score after the program. The same might also be 
true in the area of Affection. However, in the area of Control it is 
unlikely that pre- to post test scores would change, since very few of 
the program exercises or lessons deal with this area. 

Although the Heartsmart field test data were analyzed on an iiidi- 
vidual basis, they were reported as class means. Since this was the 
case, extreme high or low scores on either dimension for any area are 
**ironed out." Although scores in the area of Control are reporffed (in 
Table 23) , they are not discussed since they appear to have little rele- 
vance to program effectiveness. 

Inclusion . Tab!De 21 presents pre- and posttest mean scores on 
each of the two dimensjb^ns for each class, experimental and compari- 
son groups, at each site. 

Pretest Expressed mean scores for experimental classes ranged from 
2.77 to 5.11, and for comparison classes ranged from 3.57 to 4.78. At 
school G the two classes had identical mean scores of 3.74; no other 
pair of classes had a "match." The. greatest difference was found at 
school B (5.11 for experimental vs. 3.68 comparison). The lowest 
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Table 21 



Pre- and Posttest Results for Experimental and Comparison Groups on Expressed 



and Wanted Dimensions for Inclusion in the FIROrBC 



te 


Experimental 


Comparison 




N 


Pretest' 


Posttest 


N 


Pretest 


Poa 


ttest 




E 


W 


E-W 


E 


W 


E-W 




_.E..^-. . 


W 


E-W 


E 


W 


E-W 




23 


4.04 


3.52 


.52 


4.13 


4.30 


-.17 


21 


3.57 


4.38 


-.81 


3.62 


4.19 


-.57 




28 


5ill 


5.36 


-.25 


5.18 


5.75 


-.57 


28 


3.68 


4.07 


-.39 


3.82 


4.32 


-.50 




28 


3.64 


4.11 


-.66 


3,89 


5.18 


-1,29 


.23 


4.78 


4.74 


.04 


3.91 


5.22 


-1.30 




24 


3.25 


3.67 


-.42 


3.63 


4.25 


-.63 


24 


3.88 


5.00 


-1.13 


3.67 


4.08 


-.42 




28 


2.93 


4.32 


-1.39 


3.82 


3.93 


-.11 


24 


4.21 


4,33 


-.13 


4.21 


4.25 


-.04 




30 


2.77 


3.00 


-.23 


2.83 


4.00 


-1.17 


31 


3.68 


4.19 


-.52 


3.77 


4.96 


-1.19 




23 


3.74 


3.74 


0 


3.91 


4.70 


-.78 


23 


3.74 


4.30 


-.56 


3.39 


2.87 


.52 



ERIC 



scores for all. 14 c2:§sses were those for experimental classes at schools 
E (2-93) and F (2.77). Referring again to Table 20, it may be inferred 
that such low E Inclusion scores for schools E and F indicate that those 
experimental classes (but not necessarily all individuals) tended to 
exclude others from their activities. Scores for all other classes for 
pretest E Inclusion fell in the middle range. 

Posttest Expressed Inclusion mean scores for experimental classes 
ranged from 2.83 to 5.18, and for comparison classes ranged from 3.39 
to 4.21. Each of the experimental classes showed a slight gain from 
pre- to posttest,, which the program developer believes to be a desirable 
result. Three of the comparison classes also showed a slight gain; one 
(school E) showed no change; and two comparison classes (schools C and 
D) showed a drop from pre- to posttest. The "match" noted on pretest 
scores for the two classes at school G was not maintained on posttest 
scores: the experimental class gained .17; the comparison clasis lost 
.35. The greatest gain from pre- to posttest on Expressed Inclusion 
was found for the experimental class at school E (.89); the smallest 
gain, for experimental classes, was found at school B (.07). \ 

Pretest Wanted mean scores for experimental classes ranged from 
3.00 to 5.36, and for comparison classes ranged from 4.07 to 5.00. The 
closest "match" was found at school E (experimental 4.32; comparison \ 
4.33). The greatest difference between. a pair of classes was 'found at 
school B- (experimental 5.36; comparison 4.07). All W Inclusion pretest 
means fell in the 'middle range. ' 
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Posttest Wanted Inclusion mean scores f or ^experimental .classes 
ranged froro 3.93 to 5»75. and for comparison classes ranged from 2*87 to 
5, 22. With the exception of school E, all experimental classes showed 
a gain from pre- to postti 31: on this dimension. Three comparison classes^ 
showed a gain from pre- to post; four showed a lossc The close pretest 
match found at school E was not maintained; the experimental W score 
dropped by .39, and the comparison W score increased by .08. 

As stated earlier the relationship between that which is Expressed 
and that which is Wanted indicates the degree of intrapersonal and 
interpersonal satisfaction. ^ On Table 21, the columns headed E-W present 
the difference between the two dimensi ons . (The Wanted mean score is 
subtracted from the Expressed mean score.) The smaller the difference, 
the greater is the satisfaction. Discrepancy scores are discussed 
later in relationship to Table 23. 

Affection . Table 22 presents pre- and posttest-mean scores, on 

each of the two dimensions for each class, experimental and comparison 

/ 

groups, at each site. 

Pretest Expressed mean scores for experimental classes ranged from 
3.68 to 4.93, and for comparison classes ranged from 3.28 to 5.00. The 

r ' ' 

closest "match" between pairs of classes was found at school B (experi- 
mental 4.93; comparison 4.75), and the greatest difference was found at 
school A (experimental 4.22; comparison 3.28). All scores fe.ll in the* 
middle of the range. 

Posttest Expressed scores for experimental classes ranged from 
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Table '22 



Pre- and Posttest Results for Experimental and Comparison Groups on Expressed - 
and Wanted Dimensions for Inclusion in the FIRO-BC 



.te 


Experimental ^ 


\ Comparison 




IN 


Pretest 


, Ppsttest . 




Pretest 

> 


Posttest 




E 


W 


E-W 


E 


•W 


E-W 1 






E 


W 


E-W 


E 


W 


E-W 




23 


4.22 


4.78 


-.57 


4.57 . 


5.00 


-.43' 




21 


3.28 


4.38 


-1.10 


3.43 


4.57 


-1.1' 




28 


4.93 


5.04 


-.11 


5.68 


6.-07 


-.39 


28 


4.75 


5,39 


■ !-.64 


4.36 


5.43 


-l.p- 




28 


3.68 


5.14 


-1.46 


3.61 


5.18 


-1.57 


23 


4.57 


4.65 


-.08. 


4.65 


5.48 


-.8; 




24 


4.58 


4.96 


-.38 


4.75 


5.08 


-,33 


24 


5.00 


5.71 


-.71 


4.50 


4.63 


-.1: 




28 


3.75 


4.43 


-.68 


3.75 


3.68 


.07 


>• 

-24 


4-. 21 


4.25 


-.04 


3.92" 




-.2' 




30 


4.07 


4.90 


-.83 


3.73 


4.93 


-1.20 


31 


3.81 


3.90 


-.09 


4.16 


4.42 


-.2 




23 


4.04 


3.74 


.30 


4.48 " 


4.09 


.39 

0 


23 


3.39 


2.87 


.52 


4.52 


3.48 
— t 


1.0 



I 

I 
I 
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3.68 to 6.07, and for comparison classes ranged from 3.48 to 5.48. - With 
the exception of school E, all experimental "classes showed a gain from 
pre- to postt2est, with the greatest average gain (1.03) found at school 

Five of the comparison classes shewed a pre- to posttest gain; one 
(school E) showed no change; and one (school D) showed a loss. 

The results presented in Tables 21 and 22, and the foregoing dis- . 
cussion lead to no definitive or generalizable conclusions. Although 
the developer might argue tjiat "ideally" postt«t scores for experimen- 
tal classes should show a closer "match" than for pretest- scores between 
the two dimensions XExpressed and Wanted) for the areas of Inclusion and 
Affection, generally speaking the results do not indicate that this is so 

AlsQ, the t^eveloper might argue thaj:, for experimental^ classes , 
from pre- to posttest. Expressed scores are mote likely to increase than. 
Wanted scores. In the area of Inclusion, Expressed scores increased for 
all experimental classes; Expressed scores for Affection increased for 
four experimental- classes . Expressed Inclusion scores increased for 
three comparison classes; Expressed Affection scores increased for four-^ 
comparison classes. ' ' 

Theoretically , Wanted scores are unlikely to change. in such a . 
short time period as was covered by the field test,. The fact that, for 
all experimental classes, except the one at school E, Wanted scores in 
Inclusion and Affection increased from pre- to posttest may be due to. 
the fact that ^he pro^iratr/ f a-.Mitated students' ability to identify and 
express their needs. Wan::sd scores from pre- to posttest increased in 

* 
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Inclusion for three comparison classes, and increased in Affection f^r ^ 
five corpf^arison classes. 

Discrepancies > A psychologically meaningful discrepancy is con- 



sidered to occur when, for example, a score of 2-3 on one dimension is 
compared with a score of perhaps 6 or above on the other dimension in 
the same area of measurement (i.e.. Inclusion, Affection). A very low 
(i.e., 0-2) or very high (i.e., 7-9) score is presumed to indicate that 
the attribute measured is "noticeably characteristic" of the person. 
The critical element in^ interpreting the E-W discrepancy if; h^pw dis- 
crepant the scores actually arie/ If the scores are approximately equal, 
"the probability is that the person behaves in ways which are compatible 
with his (her) needs" (Ryan, 1971, p. 5). Conversely, "the greater the 
discrepancy between the**t^wo scores, the greater the" probability of con- 
flict and/or frustration" within the person (Ryan, 1971, p. 5). This 
interpretation is valid for the Inclusion and Affection areas. Schutz 
(1966) states: 

How one expresses himself in" the Inclusion and 
Affection areas is similar to how one would like 
to be ac'ted toward; this is not so in the Control 
area. In that area there is a clear differentia- 
tion between how one would like to be acted toward 
} and how one trie's to act toward others (p. 80). 

\ Taking the above considerations into account, a vital prerequisite 

must be met by the data: 'at the very least there must be statistical: ; 

significant discrepancies between pretest Expressed minus Wanted (E-W)' 

scores on the Inclusion and Affection scales, necessarily among experi- 
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mental classes, and ideally among comparison cla^^ses also. An examina- 
tion of the preiT.st E-W discrepancy scores, as she /n in Table 23 re- 
vealed eight statistically 3ignificant E-W discrepancies ou** of 21 
discrepancies in the experimental groups. The schools, and sea If areas 
involved were: school B — Control; school C Affection; s;:huol E — 
Inclusion, Control and Affection; school ^5 — Control and Affection; and 
scnool G — Control. Half of the significant E-W discrepancies On pre- 
test scores were in the Control area and will not be discussed, as ex- 
plained earlier. 

The magnitude of obtained 3-W discrepancies in Affec'^ion and Inclu- 
sion was relatively r.Tiall, not exceeding d.46. If the minima] ;3tan- 
dard of psychological meanin^f ulness of an E-W disci'epancy score .':5 

J / 

approximately 3.00, as suggested by Ryan (1971), then these pretest re- 



sults probably d(p not estabTish a sound or sufficient basis r'or empiri- 
cal conclusions.! Nevertheless, in tr ./interests of exploring the 
statistical resullts of the FIRO scales, attention may be turned to 
Table 23. This t^ble presents multivariate and univariate ANOVA re- 
sults on pre- to posttest E-W discrepancies on the FIRO-BC scales. In 
this table, for pre^. and posttest discrepancies ^ the plus and/or minus 
signs have no positive or negative connotations, A plus discrepancy 



indicates that the Exp^^essed score was greater than the Wanted score; 
a minus discrepancy indicates that the Expressed score was smaller than 
t^e Wanted score. Only ^our univariate test results are of specific- 
interest: school C — Affection; school E — Inclusion and Affection; 
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Table 23 



Multivariate and Univariate ANOVA Results on Pre- and Posttest 
Discrepancy Scores Between Dlnenslc i.s for each .of ti e 
Three FIRO-liC Areas for Experimental and Comparison 
Croups ut Each School 







Experimental 


Comparison 








Mean Discrepancy 


Mean Discrepancy 




SCHOOl 


FIRO-BC SCALE 


N 


rKb rUol v«tiAnuc« 




PRE 


POST 


CHANCE 


UNIVARIATE 
F-TEST 




Expressed/ Wanted 
Inc lus Ion 


23 


0.52 -0. 17 -0.69 


21 


-0.81 


-0.57 


+ 0.24 


1.15 


A 


Expressed /Wanted 
Control 


23 


-0.04 1 .22 +1 .26 


21 


1.04 


2.57 


+ 1.48 


0.06 




Expressed/ Wanted 
Al feet Ion 


23 


-0.56 -0.-!»3 +0.13 


21 


-1.09 


-1.14 


+0.05 


0.05 








MANOVA F . - 0. 


44 












Expressed/ Wanted 
Jnclus Ion 


1 

28 


-0.25 -0.57 -0.32 


28 


-0.39 


-0.50 


-0.11 


\ 

0.14 


B 


Expressed /Wanted 
Control 


28 


-1.64** -0.78 +0.86 


28 


-0.36 


-1.00 


-0.64 


3.79** 




Expressed/ Wanted 
Affection 


28 


-0. \1 -0.39 -0.28 


28 


-0.64 


-1.07 


-0.42 


0.05 








^3,50) ■ ^ 


46 












Expressed/ Wanted 
Inclusion 


28 


-0.46 -1.28 -0.82 


23 


0.04 


-1.30 


-1.34 


0.45 


C 


Expressed/ Wan ted 
Control 


28. 


0.64 1.67 +1.03 


23 


-0.78 


0.96 


+ 1.7,4 


0.53 




Expressed/ Wanted 
Affection 


28 


-1.46** -1.57 -0.11 


23 


-0.09 


-0.83 


-0.74 


|o.66 1 


MANOVA F.- , - 0.37 

(3,^3) , 




Expressed /Wanted 
Inclusion 


21* 


-0.42 ^^-0.62 -0.20 


24 


-1.12 


-0.42 


-0.70 


1.74 


D 


Expressed /Wan ted 
Control 


2U 


0.95 -0.08 -1.03 


24 
24 


-1.29 


-1.33 


-0.04 


1.45 




Expressed /Wanted 
AffeC Ion 




-0.37 -0.33 +0.04 


-0.71 


-0. 12 


+0.59 


0.65 








MANOVA F^2 " ^ 


01 
















.-1.39*** -0.11 +1.28 














Expressed /Wan ted 


28 


2. 


-0.12 


-1.16 


- « . 04 


|10. 57***1 




Inclusion 
















E 


Expressed /Wanted 
Control 


28 


0.93*** 1.03 +0.10 


24 


1.54 


1.83 


+0.29 


0.06 




Expressed /Wan ted 
Affection 


28 


-0.68** -0.07 +0. 


24 


-0.0^ 


-0.29 


-0.25 


|l.86 1 








"A^'OVA F^3^^^j - 3. 


82** 












Expressed /Wanted 
Inclusion 


30 


-0.23 * -1.16 -0.93 


31 


-r.52 


-1.19 


-0.67 


0.19 


F 


Expressed /Wanted 
Control 


30 


1.37*** 1.20 -0.17 


31 


1.13 


1.42 


+ 0.29 


0.31 




Expressed /Wanted 
Affection 


30 


-0.83*** -1.20 -0.37 


31 


-0. 10 


-0.26 


-0. 16 


|0.12 1 


MANOVA F^2 - 0.17 
* 




Ex pressed /Wanted 
Inclusion 


23 


0.00 -(..78 -0.78 


23 


-0.56 


-0.22 


+ 0.34 


3.01* 


C 


Expressed /Wanted 
Control 


23 


1 .22*** 1 .61 +0.39 


23 


0.61 


0.30 


-0.31 


0.75 




Expressed/ Wanted 
Affection 


23 


0.30 0.39 +0.09 


23 


0.52 


1.04 


+ 0.52 


0.54 








^3.40) " ^ 


.13 











* p< .05 ** p< .01 
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and school F — Affection. In the table these test; results are enclosed 
in boxes | | . 

The reason for focusing attention on these sets of results is that 
in each case, pretest E and W scores were farthest apart (i.e., signifi- 
cantly different at the p^.05 level) compared with all other E-W pre- 
test discrepancies (with the exception of the Control area which will 
not be discussed for reasons cited earlier); therefore, ^ they represent 
the strongest bases for testing whether E and W scores come closer to- 
gether on posttests. A desirable program change would be indicated by 
a reduction in E-W discrepancies. 

Indus ton . Pretest discrepancy scores for experimental classes 
ranged from +.52 to -1.39, with one school (G) scoring the "ideal" of 
zero discrepancy. 

Pretest discrepancy scores for comparison classes ranged from +.04 
to -1.12, with school^C scoring closest to the ideal with +.04. 

Experimental and comparison classes at a given site did not have 
equal discrepancy scores. The closest pair of classes was at school F 
with the experimental class discrepancy equalling -.23, and the compari- 
son equalling -.52. 

Posttest discrepancy scores for experimental classes ranged front 
11 to -1.28. 

Posttest discrepancy scores for comparison classes ranged from 
-.22 to -1.30. 

A desirable program-related change from pre- to posttest is for 
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experimental classes to reduce the magnitude of the discrepancy score. 
Such a change occurred for experimental classes at schools A and E, 
with the latter being the greater (-1.39 to -.11). 

Comparison classes showing a pre- to post reduction of discrepancy 
scores include schools A, D and G, with the greatest being D (-1.12 to 
-.42). 

The univariate test result at school E was significant; discrepancy 
scores from pre- to posttest indicate that the experimental class 
changed in the de&ired direction, while the comparison class changed in 
the opposite (undesirable) direction. 

\ The multivariate F-Test on changes in E-W discrepancy at school E 
was statistically significant (p^.05). This result is primarily due 
to large differences 'between experimental and comparison groups on 
changes in the E-W Inclusion area. Inspection of mean scores reveals a 
reduction in E-W discrepancy from pre- to posttest for the experimental 
class, and an increase in E-W discrepancy in the comparison group. The 
differences between both groups in the direction of pre- to posttest 
change account for the significant MANOVA F-Test result. 

" Affection . Pretest discrepancy scores for experimental classes 
ranged ^rom +.30 to -1.46, and for comparison classes ranged . from +.52 
to -1..09. Scores closest to the ideal were achieved by the experimental 
class at school B (-.11) and the comparison class at school E (-.04). 
The school having the closest "match" was school G (Experimental = +.30, 
Comparison = +.52). 
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Posttest discrepancy scores for experimental classes ranged from 
+.39 to -1.57, and for comparison classes ranged from +1.04 to -I.IA. 
Scores closest to the ideal were achieved by the experimental class at 
school E (-.07) and the comparison class at school D (-.12). 

Again, the desirable change is for experimental classes to reduce 
the magnitude of discrepancy scores from pre- to post. Such a change 
occurred at schools A, D and E, with school E showing the greatest 
change (-.68 to -.07). 

The^only comparison class showing a "pre- to^post reduction in dis- 
crepancy scores was school D (-.71 to -.12). 

The univariate test results in the area of Affection were of inter- 
est at schools C, E and F. At school C, although the pretest discrepanc 
score for the experimental class was significant, very little change 
occurred from pre- to posttest. Both classes at school C changed from 
pre- to post in an undesirable direction. At school F, discrepancy 
scores from pre- to posttest indicated that both classes changed in an 
undesirable direction, the change being greater for the experimental 
class than for the comparison class. At school E discrepancy scores 
from pre- to posttest indicated that the experimental class changed in 
the desired direction, while the comparison class changed in the 
opposite (undesirable) direction. 

Summary . The results indicate that only ov(i of the four group 
comparisons was statistically significantly (p^, .05) in the expocted 
direction: school E — Inclusion. The remaining threa tests were 
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either not strongly supportive of, or statistically significant enough 
to support, the desirable change. Thus, these results do not demonstrate 
that the Heartamart program has significantly influenced E-W discrepancy 
changes. 

Contingency Analysis 

Impacts of educational programs frequently depend on many condi- 
tions; This analysis will explore the possible relationships between 
outcomes of the Heartsmart program and factors within the antecedent 
and transaction categories of the evaluation model used to guide the 
overall field test. These relationships will be explored both empiri- 
cally and logically. 

Pupil sex and classroom environment are the two major factors 
wi:hin the antecedent category to be examined in relation to program 
outcomes. Partly ^. -cause several experimental teachers independently 
reported their i .•.'>ression that girls might be benefitting more from 
the program t* in boys, an analysis of program our.comes in relation to 
pupil sex djv Terences was undertaken. An attempt will be luade by em- 
pirical •;es of the data to verify or refute these teacher iu:- 
pressioi of * f 1 erential ;.x reaction to the program. 

Init.,,..' : !assroom vironment had a compelling logical iir»pvrtance 
as ..^ .'*?.tecedent factor. Findings relating to initial classroon environ- 
ment ji based on pretest scores of three scales of the My Class Inven- " 
tory (/iCI) measure; these data are not comprehensive ano therefore must 

0 

be considered with caution. 
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The contingency analysis involving program transactions and out- 
comes was carried out by logical rather than empirical means. Observa- 
tional results served as a primary source of data for this analysis. 
Specific factors to be explored in this analysis relate to the quality 
of course implementation and student behavior in the classroom. 
Sex Factor as Related to Program Outcomes 

Table 24 reports multivariate and univariate F Test results in 
reference to sex differences between experimental and comparison groups 
on the What Would You Do? and What's Happening? tests. "^Of special in- 
terest are statistically significant Sex x Treatment interaction effects 
indicating possible differential effectiveness of the program with 
respect to sex. Significant multivariate, main effects for sex were ' 
obtained at schools B, C, D, F and G, with girls scoring higher than boys 
at each school. At schools B, C and G, significant multivariate inter- 
action effects were found; inspection of mean scores indicates that ex- 
perimental girls scored highest at sch^-Js C and G and second highest at 
school By whereas comparison group boys scored lowest at schools B and 
G and second lowest at school C. In these three schools, all significant 

c 

univariate interaction effects were found only on the What Would You Do? 
test. 

Analyses of the relationship, between the factor of sex and results ' 
from the MCI and FIRO-BC were done in the same manner as above. For 
the MCI, significant multivariate Sex x Treatment interaction effects 
were obtained from the school data (F = 2.89, p<.05). Significant 
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Multivariate and Univariate ANCfVA Results of 
Sex and Sex X Treatment Interaction Effects 
on the What*fl HappentnR? and What Would You Do ? Tears 







Sample Size and Mean Score 


Univariate F-Test 


School 


Scale 


Sex 


GROUP 


EFFECT 


N 


Experimental 


N 


Comparison 


Sex 


Sex X 
Treatment 


A 


What's 
Happening? 


Male 
Female 


12 
11 


15. Al 
17.09 


10 
11 


12.30 
-'13.A5 


1.22 


0. OA 


What Would 
You Do? 


Hale 
Female 


12 
11 


8.91 
10.81 


10 

11 


6.40 
6. 45 


1.35 


0.63 


HANOVA F^2.39) " 


0.83 


0.A3 


B 


What's 
Happening? 


Hale 
Female 


U 
U 


12.42 
U.A2 


u 


11.50 
U.92 


8. 18** 


0. 56 


What Would 
You Do? 


Male 
Female 


U 


8.21 
U.50 


u 
u 


6.85 
8.85 


17.32** 


A. 63* 


HANOVA F^2.51) ' 


10.90** 


2,88 


C 


What's 
Happening 


Hale 
Female 


13 
15 


13.53 
15.20 


15 
■8 


11.53 
13.00 


3,86* 


" 0. 00 


What Would 
You Do? 


Hale 
Female 


13 
15 


3.53 
9. AO 


15 
8 


7.13 
7.75 


7.47** 


6.56** 


HANOVA F,. • 


7.90** 


3.57* 


D 


What's 
Happening 


Hale 
Female 


13 
11 


U.53 
16.27 


13 
11 


12.92 
16.63 


0. 37 


0.81 


What Would 
You Do? 


Hale 
Female 


13 
11 


13.15 
12. 5A 


.13 
11 


8.76 
10.63 


0.29 


1.13 










MANG 


VA 


^(2,43) " 


5.01** 


I.IA 


£ 


What's 
Happening? 


Male 
Female 


U 
U 


12.07 
13.35 


11 
13 


10.36 ' 
11.30 


A .29* 


^ . ZO 


What Would 
You Do? 


Hale 
Female 


U 

u 


9.50 
9.35 


11 
13 


5.09 
7.07 


0.07 


1.15 


HANOVA F^2.51) - 


0.90 


0.51 


F 


What ' 8 
Happening? 


Hale 
Female 


u 

16 


12. VI 
15. 18 


16 
15 


10.18 
10.46 


10. 10** 


1.33 


What Would 
You Do? 


Male 
Female 


U 
16 


8.21 
' 10.06 


16 
15 


7.06 
7.73 


3.66 


0.98 


"(2.59) - 


2.40 


1.31 


G 


What's 
Happening? 


Male 
Female 


13 ■ 
11 


11.76 
13.30 


10 
13 


10.70 
11.30 


0.21 


0.56 


What Would 
You Do? 


Hale 
Female 


13 
11 


6.92 ^ 
10.A5 


10 
13 


6.30 

7.00 


4.39* . 


2.99 


HANOVA F(2^,3, - 


2.73 


2.63 



♦ p ^.05 ** p ^..01 
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univariate interaction effects, all-on the Satisfaction (S) Scale, were 
found at schools A, B and C. At each of these schools, mean scores on 
the S Scale declined from the pre- to posttest. Mean scores fQr boys in 



the experimental group declined most at schools B and C, and' mean scores 
at school .A among comparison group males declined most with experi;neatal 
girls not far behind. 



significant multivariate Sex x Treatment interaction effects (F 46)~ 
2.48) at school E, and significant univariate interaction effects in 
the area of Affection at schools D and F, and in the area of Inclusion at 
school E. Discrepancy scores became smallest among the comparison group 
girls whereas discrepancies increased most among experimental group girls 
in the area of Affection. Discrepancy scores were reduced most on In- 
clusion at schools E and F for experimental females. 

The overall significant Sex and Treatment interaction results must 
be summarized in relation to each measure. With respect to the two 
criterion-referenced tests, girls in the experimental group performed 
best at three schools; on the S Scale of the MCI, mean scores declined 
most at two of three schools for boys and declined least for experi- 
mental girls at two of three schools.' Results from the E-W discrepancy , 
score analysis revealed that discrepancy scores became smallest among 
experimental girls at two of the three, field test sites. 
InJLtial Classroom Environment Factor as Related to Program Outcomes 

The principal' question addressed in this analysis is whether or 



Analysis of the FIRO-BC, E-W discrepancy score results indicated 
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not program effectiveness is contingent upon initial classroom environ- 
ment. From this analysis, it iray be possible to determine whether 
diff|erences between experimental and comparison groups on dependent 
variable measures are attributable to the effects of instruction in the 
Hear^smart program and/or to variation between groups oa initial class- 
room climate. 

First, it was necessary to determine if an empirical relationship 
existed between measures of classroom environment and program effective- 
ness. Pearson product moment correlations were computed among the 
pretest MCI S, F and C Scales the What Would You Do? , What's Happen- 
ing? tests, and the E-W discrepancy change scores on Inclusion and Affec- 
tion scales of FIRO-BC. Correlational results indicated Only a slight 
relationship among the sets of variables in the aaalysis. The magni- 
tude of most correlations (90%) indicated that less than 5% of the 
variance of the dependent variables was accounted for by the MCI pretest 
scores. Overall, the results suggested that program outcomes would 
probably not be significantly affected by classroom climates, which 
differ on the dimensions measured by the MCI Scales: Satisf action^ 

Friction and* Cohesion. * • 

Program Transactions as Related to^Program Outcomes 

The previous analyses were directed toward discovering empirical 
r^ationships between two antecedent factors, pupil sex and classro'.-'a 
environment, and observed program outcomes. Analyses were also u :^ 
taken to uncover any possible relationships between program transaction.? 
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and outcomes. Of particular interest in the "transactions" category 
were primarily qualitative factors: the quality of teacher implemerita- 
tion and the quality of student interaction occurring primarily in the 
Heart to Heart talks - 

With respect to the first factor, quality of teacher implementation, 
observational and interview data were used to rate experimental classes 
across field test sites on several variables: the amount of time the 
teacher spent on Hear t to- Heart talks and lesson "wrap-ups"; the number' 
of optional activities the teacher used; the style or approach used by 
the teacher in leading Heart to Heart talks; the type and reason for 
teacher intervention during tape-led lessons; the nature- and .degree of 
teacher innovation with respect to introducing new ideas' or activities 
not specifically called for by the program; the teacher's attitude to- 
ward students; and the* teacher's, attitude toward the program. ^ ^ 

The quantity and quality of student interaction occurring in the 
Heart to Heart talks were assessed by means of classroom obs.ervations . 
The classroom observer attempted to record a variety of student behav- 
iors: lesson pertinent verbal interaction; sharing of in-class ;.'^er-re- 
lated experiences; expression of positive and/or negative feelings to- 
ward group members; negati^^ci reactions (e.g., crying, hitting another 
pupil); number of students talking during the lesson; and the direction 

of student talk (e, g:, toward the group, the teacher, or individual 

/ 

pupils in the group). 

[ I 
Ratings given by program staff to experimental classes biT the 
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factors just described revealeid Important qualitative' differences in 

/ 

prograLA impletfientation with respect to both teacher and student^ behavior . 



However, the dif f erences ^in ratings were not helpful in explaining varia7 
tions in performance among field test sites either on the criterion- ' . 

referenced tests or on the standardized measures. It is suggested here 

/ 

that perhaps the methods of data collection and the means of analyzing 
the data were too subjective to provide insight into the classroom pro- 
cess and its relationship to program outcomes. 
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CHAPTER FIVE: SUMMARY AND CONCLUSIONS • ' ' . 

This chapter sumrriarlzes the findings of the .Heartsmart field test 
/» 

conducted In the spring of 1977. Each of the principal field test 
questions is* addr^sed in t^rn; results are summarized; conclusions are 
drawn; and recommendations are made. Finally, the overall contents of 
the .chapter are ilimmarized, and tentative claims for the program are 
formulated. i'. 

.Question 1: To what extent might the program be harmful to learners with 
respect to educational and psychological effects? 

Summary of Results . The question above was rephrased and posed to " 
the three external consultant reviewers whose reports are suramkx:ized in 
Section I of Chapter Four. Reviewers responded with varying degrees of 
directness.- Their opinions are summarized below. 

• Schutz found the program to be theoretically sound, and anticipated 
great success especially with an aware teacher. 

• Ramsdell considered it to be a great program. 

9-Agazarln believed the impact of the rogram would have a positive 
influence bn' the psycho-social growth of. p rogram^ tud en ts , and was of 

- ; ; - • X 

p^articular importance to inner-city children. . n 

^ Each consultant- reviewer identified specific lessons, situations, . 
concepts -or methods that he or she considered disadvantageous in some 
way or "another. These ^'disadvantages" relating to potential harm are 
summarlijed below. . ' 

Such*situa*tions. as running away from home, taking candy from a stranger 

V / ■ ■ , ■ • . ■ 

. ' ' 160 : • • 

178 .. - 



behaving "differently" in ways that may be harmful to others, and 
arguing aggressively with parents, although presented in the fantasy 
of Heartsmart, are not socially acceptable realities. 

• Anthropomorphis of n\ r^>»ines, such as the OOPSER, is undesirable. 

• The blackmail anc. .u ; -ent reward in Lesson 19 are undesirable, 
and are similar to tne dependency and reward in Lesson 28. 

Although Question 1 was not posed directly to persons involved -In 
the Affirmative Action Review (Section II of 'Chapter Four), the indepen- 
dent reviewers did address the question. Results from the independent 
review relating to potential harm are summarized below. 

• There is possible value-conflict for B^lack children *if they are 
taught to believe tha^ expression of feelings in any situation is 
right and will be rewarded. 

Further data relating to Question 1 are discussed in Sections III 
and IV of Chapter Four. Of particular relevance is the discussion about 
Heart to Heart talks under the subheading Interpretation^ and Disposition 
in Section III^. Results based on observational data of classroom imple- 
mentation are suiranarized below. 

• Teachers did not contradict program concepts, but did introduce varia- 
tions' of their own related to that wHich they considered *'best" for 

^ their students. For the most\part program staff considered such varia- 
tions to be harmless. 

• Teachers who were judgmental (of statements volunteered by students), 
manipula-jive (of students' desires and expressed feelings or behaviors > 



or bored (by students' accounts) may\have caused a small amounts of 
psychological harm. However, it may b^e argued that such behavior 
would be characteristic of those teachers and not solely related to 
or directlv stimulated by > Heartsmart . \ 

\ 

9 Student and teacher opinion surveys and interviews did not indicate 

thflt the program was harmful in any way. 
• Possible disadvantages as identified in the Information Pages (Appen- 
dix A), cccurred to a very slight extent as illustrated by students' 
responses to the group survey. ^ 

Conclus ions . Generally speaking the field test version of 
Hea^izsmart may be perceived as relatively harmless. However, some con- 
^^P^s could be perceived as harmful by some individuals. Also, certain 
situations or elements of the program are potentially harmful, and dir- 
ections to the teacher allow too much latitude for potentially harmful 

\ 

variations. . ' 1 .. 

Recommendations . In order to anticipate and attempt to prevent 
potential harm to students, all relevant directions to the teacher 
should be expanded and clarified, and presented inj such a way that the 
teacher is likely to be disposed to carry out the directions as intended. 

All suggestions and criticisms made relating to potential harmful- 

j 

. ness should be considered carefully: revisions to relevant lessons and 
materials used by students and teachers should be made. 

! 

. i 
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behaving "differently" in ways that may be harmful to others, and 
arguing aggressively with parents, although presented in the fantasy 
of Heartsmartj are not socially acceptable realities. 

• Anthropomorphis of t\\ r^l^ines, such as the OOPSER, is undesirable. 

• The blackmail anc ; -ent reward in Lesson 19 are undesirable, 
and are similar to tae dependency and reward in Lesson 28. 

Although Question 1 was not posed directly to persons involved In 
the Affirmative Action Review (Section II of 'Chapter Four), the indepen- 
dent reviewers did address the question. Results from the independent 
review relating to potential harm are summarized below. 

• There is possible value-conflict for B^lack children *if they are 
taught to believe tha^ expression of feelings in any situation is 
right and will be rewarded. 

Further data relating to Question 1 are discussed in Sections III 
and IV of Chapter Four. Of particular relevance is the discussion about 
Heart to Heart talks under the subheading Interpretation^ and Disposition 
in Section III^. Results based on observational data of classroom imple- 
mentation are suimnarized below. 

• Teachers did not cont\:adict program concepts, but did introduce varia- 
tions' of their own related to that wHich they considered *'best" for 
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-their students. For the i^ost\part program staff considered such varia- 
tions to be harmless. 
Teachers who were judgmental (of statements volunteered by students), 
manipulative (of students' desires and expressed feelings or behaviors^ 



present rn the portrayal of female and non-white characters, 

Recommendations . All illustrations and related taped voices should 
be determinate in terms of race or ethnicity. A balance of superordinate 
and subordinate roles and positive and negative characteristics should 
be achieved across sex and race or ethnicity. Qualitative representa- 
tion (nature and extent of participation) should be considered as well 
as quantitative representation by sex and race or ethnicity. 
Question 3: Were all essential parts of the program implemented as " 
intended ? 

Summary of Results . Implementation is discussed at length in Sec- 
tion III of Chapter Four. 

• All structured lessons, i.e., the tape-led lessons and Heart to 
Heart talks, were implemented in tlie given sequence, without omissions 
or major changes, at all sites. 

• All imperative directions were followed at al] sites. 

• All guiding directions were followed at all sites with some slight 
variation influenced by teachers' dispositions. 

• Discretionary directions were followed at all ^ites with considerable 
variation among classes. 

•Where little or no direction was given, teachers ' initiated innovations, 
the greatest of which was the "wrap-up" at the end of tape-led lessees. 

• Disposivtional differences in implementation were most apparent in 
Heart ^to Heart talks, and in the ntanber and nature of optional teacher- 
led activities. * 
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Co nclusions > The program was Implemented as intended at all sites, 
and was not distorted by variations and innovations. However, it is 
important to remember that. these results are based on a field test 
situation in which each class was visited by an observer about once 
every four lessons; without such observation ±t is possible that teach- 
ers would introduce greater variations and/or innovations. 

Recommendations . Clarify all directions to the teacher. Provide 
for ways in which the teacher may conduct a 'Vrap-up" after a tape-led 
lesson. Provide greater clarification of the rationale for the program 
and its concepts and goals, so that teachers will be more likely to 

m 

introduce appropriate innovations, and to appreciate -the kinds of vari- 
ations most suitable to the program. 

Question 4: Was the program educationally appealing and worthv/hile to 
teachers and students? 

Summary of Results , Section IV of Chapter Four discusses results 
related to this question." General findings are summarized below. 

• The majority of students enjoyed the tape-led lessons and games, 
finding the latter easy. 

• Teachers rated the concepts of the tape-led lessons important to their 
students' lives, and found the program easy to administer. 

• The majority pf students enjoyed the Heart to Heart talks, found them 
interesting and wanted to talk more, 

© Most teachers found the concepts of the Heart to Heart talks to be 
worthwhile and easy to administer, although there seemed to be some 
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doubt as to the value of the last .Heart to Heart talk (Lesson 29). 
• Students and teachers reported having enjoyed the optional activities 
However, data indicated that fewer optional activities were used in 
• last two-thirds of the program than in earlier lessons* 
?f •". : students eajoyed working in groups, but there was some disparity 
iiw.'u J/)d:lvidaals, groups, and classes relating to group roles, espec 
:.Ll^ rbat <<? leadership, 

Conc^.iSions. Generally speaking the program was appealing to at 
least 75% of tiie students with little difference among ''.sites and was 
appealing and considered to be educationally worthwhile by the teachers 

Recommendations . Although Heartswai:t is not a cognitive program, 
the question of Ciasiness of the tape-led games should be considered, 
balancing students' perc^^ptions of achievement with the program objec- 
tives.., Perhaps some tape- led gam^s should be more challenging. 

The Heart to Heart rr-^lks should be structured to allow for maxi- 
mum participation of Sw .idents. Lesson 29 should be revised, perhaps 
to sum^narize some of the :'iajor program concepts. 

Optional activities shoL ^d be reorganized and/or added to, so 
that tbc:3e presented in Units II and III appeal to the teachers, and 
so tha^. over:- vl there is a wide variety of activities from which a 
teacher may choose" 

Group skills and group roles, especially leadership, should be 
taught a'" "^/or discussed. 
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Question 5: How well did students perform on Inst ructional tasks of 
^h q program, which reflect achieveinent in understanding and knowledge 
acquisition? 

Summary of Result s > In Section V of Chapter Four, discussing 
student achievement, reference is made to Gamepages and Reviewpages. 

c 

Th^ former were d signed to allow for practice of affective concepts. 
The latter were desi3ned to measure cognitive comprehension of those 
concepts. In light ol the question as phiased, only the results relat- 
ing CO the Reviewpages are discussed here. 

# Analysis of Reviewpages from a r^nd:.-.: sub-sample of students in 

Heartsmart classes showed that approximately 95% of the questions were 
answered correctly by all scuccnts. 

Cone lusions . The Reviewpages probably did not Accurately assess 
''true" le\e)s of studci^t achievement in. understanding what the program 
taught in tlie tape- led lessons. Th3 questions asked on the Reviewpages 
were probably too easy. Since the Reviewpages were not administered 
to non-i^/eartsjnart-insr.ructed students, it is impossible to judge achieve- 
m'lat in a comparative way. Attempts to construct k "mastery test" for 
the program, as described in Chapter Two, showed that non Heartsmart- 
Instructed studev. could perform equally as well as Heartsmart students 
on the R - Va.ewpages . The evidence presented in this study leads to the 
conclu^^-on that al.nost all Heartsmart pupils can grasp the main ideas 
pr'-jjented Iti the tape-led lessons, although other factors (e.g., general 
knowledge, common sense, the effects of socialization in our society) 
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besides Heartsmart instruction may account for student achievement 
results. 

Recommendations . Delete the tape-led review and the related 
Reviewpages in the Student Journals. Provide teachers with review 
questions of varying levels of difficulty to be used as part of the 
teacher-led "wrap-up" at the end of each tape-led lesson.' 
Question 6: How effective was the program in terms of accomplishing its_ 
major instructional goals and objectives? 

Summary of Results . Effectiveness data are presented in detail 
in Section VI of Chapter Four. 

• At all field test sites, Heartswart program students demonstrated sig- 
nificantly higher performance ^than comparison group students in crite- 
rion-referenced tests. 

• The specific areas in which Hearzsmart program students appeared to 
excel over comparison group students were: identifying feelings and 
behaviors and inferring feelijngs from behavior in others,; identifying 
ways in which people, may behave when they have certain feelings; deter- 
mining feelings in others by asking; and expressj-ng^needs and desires 
and taking actions to accomplish them. ^ . 

• Heartsmart program students ^reported learning interpersonal skills 
"consonant with most .of (he instructional objectives of the program,' 

excluding perhaps .ne following objective: expressing individual 
needs and desires and takin^^ appropriate actions to accomplish them. 
(The only data available for this objective, other than the 

■168 , . 
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criterion-referenqed tests, were subjective judgments made by observ- 
ers, who could report only superficial and general observations. Al- 
though it is possible that within small groups this objective was 
achieved, there is no related data since the small group interaction 
was not closely observed or evaluated, ) 

• Program students reported having expressed their feelings to others" 

e 

more often after the program than before. 

• Several teachers reported instances in school where Heartswart program 
students used interpersonal skills outside o'f class situations. 

• The classroom environment of Heartsmart classes did not consistently 
become more cooperative and satisfying relative -to comparison classes, 
based on standarr*ized measures; students and teachers reported a ten- 
dency toward increased cooperative behavior and less friction in the 
experimental classes from the beginning to the end of the program. 
(Observational data, for the most part,, supported teachers' findings). 

o Expressed-Wanted discrepancy scores in the areas of Inclusion and Af- 
fection amoJig Heartsmart program classes did not become significantly 
smaller from pre- to posttest compared with non-Z/eartsmart-instrueted 
classes. 

Conclusions . The Heartsmart program appears to be successful in 
achieving the objective of teaching students ceveral specific interper- 
sonal skills, and possibly of facilitating the application of these 
skills to out-of-class peer and adult interpersonal situations.. The re- 
sults from the criterion-referenced tests provide strong statistical 

o 

O 
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evidence of program effects in the expected direction; it is difficult 
to find a plausible alternative explanation for the observed results on 
these tests given the overall strength and consistency of those results 
across field test sites • 

No definitive conclusion is reached with respect to the program's 
ability to improve the psychological climate qf the classroom. Results 
from" comparative group data on the MCI were generally contradicted by • 
student self -report and teacher observations/perceptions of student 
classroom, behavior over time. The discrepant findings might be partly 
explained by the low reliability of measurement in the MCI scales. Also, 
it is possible that the positive scales of Cohesion and Satisfaction 
(of the MCI) were negatively inj^luenced by students' experiences within 
their small groups if these experiences were unsatisfactory, and that 
.the negative sca'.e of Friction (of the MCI) was similarly influenced. 
However, the nature and extent of data collection for student self-re- 
ports did not allow for comparative analysis with MCI results; so such a 
conclusion must be extfemely tentative. 

The quality and quantity of evidence presented in reference to eval- 
uating the program's ability to produce a better match betvzeen students' 

'J 

expressed and Wanted interpersonal needs in the areas of inclusion and 
affection behavior were insufficient to 'reach a definitive conclusion. 

Recommendations , Allow for greater practice of interpersonal skills, 
possibly through the use >of optional teacher-led activities. Decrease 
the likelihood of friction within small groups by {Providing training^ 

. ■ , 170 , 

■ ,' 188:' 



in group roles and skills. Allow for evaluation by providing the teach- 
er with criterion-referenced tests, scoring keys, and directions for 
administration and interpretation of results. 

Question 7; -Wh at factors among antecedent' conditions and program trans- 
actions relate to the nature and extent of program effectiveness? 

\ ' ' ' " 

Summary of Results , 
m Significant multivariate and univariate Sex by Treatment interaction 
effect's were found on the What's Happening? an: What Would You Do? 
tests at three of seven field tfest sites; girls\n the experimental 
group obtained tKe highest mean scores. 

• Significant univariate Sex by Treatment pre- to^^posttest interaction 
effects were found on the Satisfaction scale of the' MCI at these three 
schools; scores amoiig boys in the experimental groups declined most 

at all three schools. 

• Significant univariate Sex by Treatment interaction effects were * 
found at three schools., on ilxpressad-V/anted discrepancy change scores 

. on the FIRO-BC Mfection and Inclu,??ion scales^ the largest reduction 
, in E-W discrepancy scores occurred among experimental .girls at two ' 
of the thre^e schools. ^ " '■ 

• Correlations between pretest MCI scales and performance on the What's 
Happening ? , l-fhat Would You Do? and the FIRO-BC E-W discrepancy change 
measures were very low. 

• Although variations in program implementation occurred, they seemed to 
have very little impact of prograh effectiveness as measured by stan- 
dardized and criterion-referenced testSe 
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Conclusions . The significant relationships between pupil sex and 
program outcomes at some field test sites show a tendency toward differ 
^ntial impact of the program according to sex. This conclusion applies 
more strongly to those program outcomes measured by the criterion-ref- 
erenced tests'. Girls in the experimental classes outperformed any 
other combination of sex by group, suggesting, perhaps, that the pro- 
gram may be more effective with girls. This matches the impression^s 
of at least three experimental cdass teachers who suspected indepen- 
dently that girls were benefitting more than boys from the program. 

■ \ ^ 

9 

Experimental group boys, however, also did better on the average than 
boys in comparison classes. / / 

Educational outcomes of the program were not found to be related 
to tfie pre-existing social climate of the classroom. This finding 
V70uld suggest that highly negative (e.g., hostile, uncooperative) class- 
room social climates should not deter teachers from U3ipg the program. 
Wliatever benefits accrue to students from being eiq^ose/J to the program 
do not af^pear to be inf Juenced by the level of perceived hostility or 
dissatisfaction among students in the classroom. 

It is - interestirig to speculate on finding no meaningful relation- 
ship between the nature and. extent 'of variations in implementation 

and program outcomes. The potential for impact may be inherent ia^the 

— - - ■ ■ . ^ 

prepared audio tapes whicV. account for 24 of the 30 lessons in the 

course. The quality of implementation may, therefore, not be so much 

a function of the teacher. Differenps in the quality of implementing 
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this program n^igliL be less important than for other educational programs. 
However, 4t is important to note that these findings and conclusions 
are based on the somewhat artificial situation of a field test during 
which teachers and studc its knew that a progra^A observer would be pre- 
sent for approximately 25% of the lessons. This factor may \ well have 
influenced teachers and students, possibly by preventing or curtailing 
any strong variations outside the given program guidelines. It is.also 

important to bear in mind that field test teachers were selected or in- 

j 

vited by their school principals, were experienced, and participated in 
t:he field test willingly. 

Re c omme nd a t i ons . The pre-field test teacher training and the sup- 
port and ^si stance provided by program staff during the field test may 
;not alw^ays be provided if or when the program is made publicly available. 
^Therefore, a suitable substitute should be designed so that teachers 

r ■ ■ " 

will be likely to implement the program as intended, and all information ' 

I ^ ■ ■ . 

land directions provided for teachers .should facilitate desirable outcomes. 

I ■ . ' ^ . 

} " • 

pverall Conclujgj^ons - . 

j . . . • ' ^'J — 

It is appropriate to conclude 'this chapter with statements about 

{ 

the major strengths arid weaknesses, of the Heart smart program.'' 

" [ • \ I ■ ' ' ^ . ' 

o The Heartsmart program is likely to provide an appealing, educational- * 

" ■ 1 ' 

ly relevant and worthwhile experience to fourth grade children - r^ard-. 

less of ethnic/ racial and/ or socio-economic backgr^STni. ' 

The Heartsmart program as a whole can be Easily admiriistered by the 



teacher,, although- certain activities require special group process 

I skills to achieve maximum results, * 7^ 

■ - /■ < . , ■. . 
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• The Heart smart program will teach fourth grade students to be more 
proficient in: identifying feeling^:* and inferring feelings from the 
behavior of others; identifying ways in which people behave when 
4:hey have certain feelings; determining feelings in others by asking; 
expressing needs , and .desires- and taking appropriate actions to accom- 
plish t^iem; akid expressing feelings. ' 

The above claims represent the minimal number of areas of major 
program strengths as determined in this field test* In its current 
^brm, however^, there are two deficiencies in the program. 

One deficiency relates to sex and racial/ethnic bias ir the pro- 
gram materials,, wherein female and non-white characters tend to be por- 
trayed in stereotypic terms. A fully documented account of revisions 
made to tha^program based on the field test results is presented in 
Chapter Six. major part of the revision work was designed to remove 
the social bias in the materials. The next and final version of the 
Eeartsxt\art program should reflect a more positive and realistic view 
toward pex and racial/ethnic differences as depicted in the characters 
'6f---irhe program.* * ' 

The second deficiency relates to the nature md extent o.^ informa- 
"tibn directed to teachers, to imztease their understanding of program 
goals, 'and to facilitate the application of appropriate teaching methods 
and^ skills. This suggests rifiorganization, expansion and revision of 



-the, teacher ' s manual. 
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CHAPTER SIX: PROGRAM REVISIONS 

This chapter describes the revisions made to the program on the 
basis of findings discussed in the praceding chapters of this report, 
and taking into consideration the frcl ♦'hat subsequent use of the, pro- 
gram is unlikely to include the training and assistance provided for 
the teachers in the field test. Thus, where practi<.al, suggestions and 
recommendations made in Chapter Five have been incorporated^ in the final 
revised version (RV) of the program in an attempt to pro ide a program 
which is effective, acceptable and appealing, and which can be used by 
teachers unfamiliar with tae program or its goals. 

Each of the three sections of this chapter relates to a component 
or major area of the program, namely: the teacher's manual; the drama- 
tized story and its characters; and student^materials. In each case 
the RV and field test version (FTV) aj^e compared and "reasons for revi- 
sions are identified and discussed. - 

I." THE TEACHER'S MANUAL 

For a program such as Heartsmartj two questions are important in 
the development of the teache,r's manual. They are: (1) What does the 
teacher need to know? and (2) How can the necessary information be 
presented in such a way as to facilitate reference and application? In 
addition to these two general questions, a third is also important in 
as much as the final version of the manual must stand alone; technical 
assistance or training may not always be given if or when the program 
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is 'marketed. This third question is (3) What materials should be in- 
eluded to facilitate self -.^raining of the program teacher (s)? 

An outline of the major sections of both the FTV and the RV manual 
is presented in Table 25. The table shows that the manual was reorgan- 
ized into five sections. Each of those • sections is discussed. 
Introduction to the Teacher's Manual 

Material considered to be introductory in nature was found in foui 
sections of the FTV manual. Since teachers found this organization con- 
fusing, reorganizat-J ,1 was necessary. A summary of the RV Introduction 
is presented below, with notes referring to the nature and extent of 
revisions made. 

Program description . Based 'on material from the FTV sections titled 
"program organization" and "program goals," this section was revised in 
order to clarify the content. 

The teacher and the progra m. The related- FTV section titled "using 
the teacher's manual" presented a summary of the confe-ents of the manual 
and was found on page 11. In the RV this section advises teachers to 
follow a five step process to familiarize themselves with the program 
and related tasks. This revision was suggested orally by a field test 
teacher. 

Description of materials . The: FTV manual did not identify or des- 
cribe materials. This section is new content included tc facilitata 
program implementation. 

Theoretical rationale . • Based on material from the FTV manual Unit 
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Table 2^^ 

Comparison of the Contents of Two Versions of the Teacher's Manual 



Field Test Version 



Revised Version 



Introduction ^ 
Program {;oals; 
Program organization; 
The teacher's role (including - 
scheiduling, administration, 
optional activities, songs. 
Heart to Heart talks); 
Evaluation ; 

Using the teacher's manual; 
Trouble shooting - about materi- 
*ils, student behaviors, parents. 

Unit I: 



Xntrodu':tion - unit goals and 
objectives; 

General procedures - administra- 
tion; 

Heart to Heart discussion 
guidelines; 

Sunimc.ry of Unit 1 procedures; 
Lesson notes for Lessons 1-12. 

c 

Unit II: 



Introduction - overview, theo- 
retical rationale for Units II 
and III, goals and objectives 
for Unit II, forming the groups,' 
group leadership and leadership 
training, classroom organization, 
songs ; 

General procedures - administra- 
tion; 

Summary of Unit II procedures; 
Lesson notes for Lessons 13-21. 

Unit III: 



Introduction .- overview, goals 
and objectives for Unit III, group 
leadership^ songs; 
Summary or Unit. Ill procedures; 
Lesson notes for Lessons 22-30. 

.Appendix A: 

Tape script for group leadership 
training sessions. 

Appendix B: , . 
Handout pages for activities, 
son'" sheeCi^, instructions to 
suL^Litute teachers, and a 



group roaster page. 



- 



Introduction : 

Program description; 

The teacher. and the program; 

Description of materials; 

Theoretical Rationale; 

Goals and objectives; 

Administration ; 

Heart to Heart discussion guide- 
lines; 

oup formation; 
Group leadership ; 
Group training lesson; 
E- .:luation. 

Lesson Notes: 



Lesson notes for all lessons. 



General Activities: 



Vocabulary 
Study Center 
Homework 
Inclusion 
Control 
"Affection 
Stories 
Songs 

Follow Through 



suggestions for 
activities for 
each given 
sub-heading 



Workshop Guide: 
Overview; 
Rationale ; 

Moderating the Workshops; 
Wovkshop ill - Self-Training Session 
Workshop i'? - Sharing Session 
Workshop //J - Planning Session 

Handouts: ^ 



Handout pages for activities; song 
sheets; instructions to substitute 
teacher ^ script for group train- 
ing lesson; information pages; 
copies of evaluation measures. 
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II Introduction, this section was expanded in order to clarify concepts . 
questioned t>y field test teachers. 

Goals and objectives . Although content remained relatively un- 
changed, FTV material was reorganized and condensed, and is presented as 
one complete section of the RV manual Introduction, rather than in sev- 
eral sections with repetition. 

Adminis tration . This section is similar to the administrative 
directions given in various parts of the FTV manual, and includes ^minor 
additions. 

Heart to Heart discussion guidelines . This section is different in 
both organization and content. Organizational changes were made to 
clarify content and avoid repetition. Content changes relate to the 
teacher's role and include discussion of productive and counter-produc- 
tive attitudes and behaviors. Suggestions are given for varying the ^ 
presentation or methods of response for tha Heart to Heart talks. Some 
of the trouble-shooting material from the FTV is included in this<::5 
section. #» 

Groups . In the FTV manual, group formation guidelines were pre- 
sented in the Unit II section. In that same section leadership was 
discussed, the teacher being directed to orient student group leaders by 
playing a leadership training tape, and helping those student leaders to 
follow the directions given on the tape. These directions related pri- 
marily to administrative duties such as distributing materials. 
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The leadership tape became obsolete when the materials were rede- 
signed so that group administratioti was no longer a concern. 

Criticisro of the methods used for group formation, and strong sug- 
gestions for group training rjather than leadership training indicated 
that wajor changes should be considered • ' 

In the RV manual, the group formation methods are modified. Leader 
ship is discussed with a stronger emphasis on traits facilitating group 
maintenance rather than on administration. The leadership tape has been 
replaced by a teacher-led lesson for all students focusing on group 
skills and behaviors. In this lesson volunteer students dramatize a 
script (included in the Handouts section), role-playing imaginary stu- 
dents involved in a Heartsmart lesson. Students in the. audience iden- 
tify positive roles and behaviors dramatized, and discuss them with the 
teacher. 

Evaluation . In comparison to the FTV discussion titled "How do 
you know the program is working?" the RV section on evaluation is more 
formal. The subjective evaluation methods discussed in the FTV have 
been revised and expandsd. Descriptions of What's Happening? and 
vniat Would You Do ? (two criterion-referenced Treasures used in the field 
test) are given, together with guidelines for their use, scoring keys, 
and guidelines for interpretation of the results. Copies of the mea- 
sures are included in the Handouts section of the RV manual. No mea- 
sures were discussed or used in the FTV. 
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Lesson Notes 

In both the FTV and RV manuals, lesson notes pages consist of notes 
to the teacher, summaries of the taped story and tape-led activity, re- 
duced copies of student materials, and suggestions for teacher-led acti- 
vities relevant to the given lesson. Lesson notes pages to the Heart 
to Heart talks, in both versions, consist of teacher's notes and main 
points and questions to focus the discussion. 

The FTV tea cider's notes presented lesson objectives, materials 
neednd, new vocabulary, and special instructions, where necessary. This 
format is retained in the RV. Review questions are also included in the 
RV. 

In the field test each lesson was reviewed by the teacher-on-the- 
tape, who asked five comprehension questions requiring "yes" or "no" 
answers. Students responded by checking appropriate responses on Re- 
viewpages. Correct responses were 'then given by the teacher-on-the- 
tape. The content of most review questions in the RV is similar^ to 
that of the FTV but questions are phrased differently, some new 
questions are included, and all are noaed by the classroom teacher 
and discussed orally. The administration section of the RV Introduc- 
tion discusses the review questions and acceptable student responses. 
This revision was made for several reasons. In th<i field test classroom 
teachers wanted to participate in the tape-led lessons and often re- 
viewed each lesson after the tape had finished. The students found 
the review boring and the questions too easy. Analysis of student 
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achievement supported the students'' latter reaction, Teachc. ? and 
RBS staff considered oral review preferable to a pencil and pap^r 
review. 

Two relatively minor changes in the RV teacher's notes are (1) the 
explication of points of difficulty with suggested procedures for alle- 
viating or eliminating the source of difficulty, and (2) the addition 
of "special notes" which are used to elaborate upon concepts or situa- 
tions in the story. For example, one note suggests that the teacher may 
differentiate between reality and fantasy, and explain that although in 
the story Jack accepts candy from the Far Out Fish, in reality students 
should not accept gifts from strangers. 

Summaries of the taped story and tape-led activities, and copies of 
student materials were revised in accordance with relevant changes ±h 
those areas. 

Suggestions for teacher-led activities were modified to avoid 
repetition across lessons and to ensure that activities suggested are 
lesson-specific. (See also Activities below,) 

The teacher's notes for the Heart to Heart talks were changed in 
format to enable a clearer and more concise presentation of the pro- 
cedural requirements for each talk, and to include suggestions to the 
teacher to refer to the discussion guidelines given in the Introduction 
of the teacher's manual to determine possible variations,^ The RV Heart 
to Heart teacher's notes are organized under the headings: objectives, 
preparation and directions. The main points and questions are sonjewhat 
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modified. .7 

Lesson specific activities* The optional teacher-led activities 
found in the FTV manual underwent several organizational changes. 
In the RV manual activities are included with each set of lesson notes 
(as they were in the FTV> and have been labeled "lesson specific," to 
differentiate them from those grouped in an entirely new section of th^ 
manual containing general activities • 

The lesson specific activities in the RV manual, included with 

the lesson notes, consist of those from the FtV which 'were not removed 

to the general activity section, and relevant additions. The original 

I 

presentation divided the activities ^into those which were designed for 
individuals, and those designed for groups, but such classification ' 
was sometimes erroneout. and frequently misleading. In the RV lesson 
specific activities, the classification is not used. 

Table 26 indicates that there were 152 lesson specific activities 
presehted^ in the FTV. (Forty-two of those activities were repetitions 
equalling 6 different activities, each repeated on average 7 times,) 
There are 120 RV lesson specific activities, made up of 64 unchanged 
or slightly revised FTV, activities and 56 nev: activities. Those FTV 
activities not included as RV lesson specific activities were om-'.tted 
for one or more of the following reasons: (1) they were in the group 
of 42 repeated activities which were general across the program rather 
than specific to a given le^sson; (2) they were unappealing to teachers 
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Lesson Specific Activities as Presented in 
Two Versions. of the Teacher's Manual 
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and students; and/or (3^ they were essentially minor constituent parts 
of other, activities. 

General Activites " . - 

The General Activities section, is a new addition to. the teacher's 
manual, and was considered necessary for organizational and enrichment , 
purposes. More specifically,, the reasons for developing a separate and " 
and comprehensive activities section were: 

• ^,to select from the lesson specific activities those which apply 

equally well to all lessons, and, thus reduce repetition; 

• to incorporate in, the RV, activl-ies used and recommended by field 
test teachers; ' , 

• to organize some of the FTV activities into, a meaningful, framework 
for easy reference and appropriate use; 

• to provide activities which require parental involvement; 

• to provide for transfer of concepts to other curricula; 

• to provide activities designed for active rather than passive learning; 

• to help facilitate freedom of expression positively and constructively; 
o to enhance program concepts during and after the conclusion of the 

program through app3 Icacion^j-of 'Concepts in new and different ways 
outside the classroom. • 

The 92 activities suggested -in the General Activities section of 

V 

the RV manual consist of 15 FTV activities condensed ■fi:om^the 88 FTV 
activities de"leted from the lesson specif ic, group, plus 77 new activities 
Table 27 briefly describes^ the contents of the General Activities 

i \ • . 
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Table 27 



General Activities as Present^ed in the Revised 
Version of the Manual, with Reasons for Additions 



' ^ Activities 



Reasons 



Number 



Vocabulary - words and phrases 
with' prograiD-specif ic mean- 
ings, and/or abcve_ 4th grade 
level,* 



Study Center - methods Suggest- 
ed for organizing a s^tudy* cen-^ 
ter., plus Activity directions 
for students to work Injiividu- 
ally.-" ' ^ p ^ 



Homework - three activity sug- 
gestions for each unit,' each 
activity designed to* involve 
parents and/or .family members. 



][nclusion - activities classi- 
fied in the areas- of: coopera- 
tion; shyness and loneliness; 
showing off and getting atten- 
tion; and selfishriess and 
being aloner* 



Control - activities relating 
to leadership and group roles 
and skills. 



Listed in lesson notes in fhe 
FTV, vocabulary was not defined, 
and ne related ac'tivities^ere 
suggested. Revision designed 
to increase »coraprehension. 

Six-, of ^tKe activities were in 
the ^TV; eight;J^were developed 
either by field test teachers 
or program staff . ^ Two field j 
test teachers .organized study 
cent;ers^, and others were inter- 
est eel. in the idea. »^-\ 

Two of the activities were in * 
the FTV; seven were developed 
either by field test tjeachers 
or program staff, ±n response 
to needs identified by consul- , 
tant reviewers and/or teachers. 

Designed by program staff, tlie 
activities reinforce positive 
aspects of Inclusion, using a 
variety of methods -and skills, 
and Vera deVelo^ped in response 
to suggestions from teacher;s 
and/or consultant reviewers. 

Designed by program staff, these 
activities wei^e developed in an 
attempt to Tresolve or avoid con- 
flicts observed in small groups 
during the field test. • 



*Grade level deterrr.ined according to Revised Care Vocabulary 
(Educational Development Laboratory, McGraw-Hill, 1968). 
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Table' 27 contd.. 

^ 



Activities 


Reasons «. 


Number 


Affection - activities classi- " 
fied in the areas of: friend- 
snip, aggression; and. seir— 
esteem. 


Designed by program staff, 
these activities reinforce 
positive aspects of Affection, 
and allow for acceptable ways^ 
^ in which anger or aggression 
may be expressed, and were 
developed in response to sug- 
gestions made by consultant 
reviewers and/or teachers. 


21 


Stories - an annotated bibli- 
ography of 20 short stories 
relating to program objectives, 
to be read by the teacher to 
the students^ 


Selected by teachers or pro- 
gram staff, these stories 
re^-nforce program objectives. 


I 


Songs - three^ suKKestions for 
ways in which program songs 
(presented on song sheets and 
on an audio cassette tape) may 

be used. ■ ^ . 

■ " «^ ■ 


•Suggestions implied in the FTV 
were clarified; 




Follow Through - suKgestions 
for sharing program concepts ^ 
or following -through after 
the program has been completed, 
classified under the headixigs: 
Adopt a Class; School Project.; 
•and Community Project. 

/ . 


Designed by program staff and 
field test teachers, these 
.: activities were developed In 
responise to. suggestions made 
by consultant reviewers and/or 
teachers. 


20 




t 

>. 
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section and presents reasons for additions. 
Workshop Guide 

•J. 

As stated in the introduction to this section, it was considered 
necessary to provide materials for program teachers that could take the 
place of the training and technical assistance provided during the field 

test. The. Workshop Guide is {herefvre a new addition to the manual, and 

\ 

contains directions for thrae workshops^, '»each having a different empha- 
sis and directed toward a different group of participants, but all de- 
signed to facilitate program teachers' understanding and implementation 
of the program. Guidelines for each workshop are' preceded by scenarios 

designed to- increase the teachers' comprehension and confidence, and to 

y 

facilitate implementation of the workshops. 

Workshop #1 . This self-training session is to be conducted be- 
fore the program Us introduced into the classroom, and is designed for 
the school principal and program teachers to facilitate comprehension 
of program concepts, goals, and guidelines; to provide opportunities 
for familiarization with program materials; and to involve participants 
in experiential activities related to the program. Although it is 
possible for teachers to familiarize themselves with the program in 
isolation, it is preferable for teachers to experience some program 
activities and to discuss concepts and possible problems with their 
peers. The moderator of the workshop is to 'be a teacher planning to 
use Heartsmarty or a person with training- and experience related to the 
program goals. 
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Workshop //2 .' This sharing session :^.s directed at those people 
most likely to feel the impact of the program through the behaviors of 
program students in situations outside the classroom (e.g., parents, 
non-program teachers) . Since Heartsmart teaches interpersonal skills | 

which necessarily are used with people outside of the classroom, it is | 

i 

important that these "outsiders" (at least the most important outsiders) 
understand the intent and \iature of th^^jrogram so as not to misinter- 
pret a child's motivations when applying program skills. The program, 
teaches behaviors that, traditionally, are not expected from fourth 
graders and which, .consequently, are difficult to accept or comprehend. 
For example, an honest request from a student to know specifically how 
a teachei" or parent is jS^eling in a conflict situation^ may easily be 
misinterpreted as insolent. Workshop //2 attempts to educate parents 
and other individuals as to what the Heartsmart program teaches 
and, more importantly, what types of behaviors to expect from students 
participating in the program. In addition, it attempts to develop in 
these individuals a desire, as well as the knowledge needed, to promote 
Hee/rtsmart skills at home and in other situations. The workshop is to 
be conducted midway through the program by a program teacher. Parents, 
non-program teachers, and other individuals are to be invited to attend. 

Worksho_£ //3 . This planning session is designed to be conducted . 
toward the end of the program to allow program teachers and other 
faculty to evaluate the impact of the program and to plan ways In which 
positive results may be fostered and continued. It is considered im- 
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portant that the skills and behaviors taught in the program be reinforced 
in both- school and family environments, after program completion. The 
workshop is intended to serve as a planning session for ways in which the 
school and community may facilitate the continuation of student practice 
of Heartsmart concepts. The moderator of the program is ^to be a program 
teacher and/or the schovil principal. Program teachers, the school 
principal and other faculty are to participate; parents and district- 
level personnel may also be invited. 
Handouts 

. The "Handouts" section of the RV manual is essentially sjmonymdus 
with the appendices found in the FTV manual (see Table 25). Certain 
items have been deleted or added td reflect the content and organiza- 
tional changes made to the other sections of the manual. Table 28 lists 
FTV and RV handouts and includes revision comments. 

II, THE DRAMATIZED STORY 

The story, dramatized in 24 audio tape installments , presents the 
program concepts. Needed revisions were determined by the results of 
the affirmative action review and comments and suggestions made by con- 
sultant reviewers (see Chapter Five) . Revisions are discussed under • two 
headings: revisions to the story; and revisions to program characters. 
Revisions to the Story 

All revisions to the story are relatively minor. They are sum- 
marized in Table 29, which lists changes made and reasons for those 
changes . 
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Table 28 



Comparison of Handouts in Two Versions of the Manual 



FTV Handouts 


, RV Handouts 


Comments 


3 story summaries with 
missing words to be filled 
in by students. 


' .FTV handouts 
abbreviated and 
rewritten. 


FTV readability 
averaged 8th grade. 
RV readability 
averages 3rd grade.* 


3 word searches. 


FTV handouts reduced 
from 3 pages to 1 page. 


Redesigned to reduce 
teacher's work load. 


8 song sheets . 


FTV handouts redesigned. 


FTV words and music 
too small for students 
to read. 


^Instructions to 
substitute teachers. 


FTV handoufs revised. 


Revisions relate to 
redesign of materials. 




Two dot-to-dot pictures 
of story characters. 


New handout, for fun. 




Script for gr6\ip 
training lesson. • 


New material, 
necessary to teach 
students group roles 
and behaviors. 




Tell-a-gram 
Happygram 


New material to be used 
by students, parents, 
and workshop 
participants. 




Information pages 
describing tha 
program. 


New material to be 
used by workshop 
participants. 




Copies of criterion- 
referenced measures: 
What's Happening? and 
What Would You Do? 


New material to allow 
for evaluation. 


Crossword puzzle. 


FTV handout deleted. 


FTV design and content 
irrelevant to RV. 


Group roster. 


FTV handout deleted. 


Unnecessary. 


Leadership training 
script. 


FTV material deleted. 


FTV script replaced 
by, group training 
script and lesson. 

1 



^Readability determined by the Readability Graph, Edward Fry, Journal of Reading , 
April, 1968. . 
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Table 29 

Comparison of Story Elements in Two Versions 
of the Program with Revision Comments * 



r 

Lesson it 


FTV Story 


RV Story 


Comments 


1 


Fast heartbeat related 
to anger and f^ar.'^ 


Fast heartbeat related 
to any strong emotion 
or physical effort. 


FTV inaccurate and 
»nis leading. 


9 


Computer broke down 
emotionally and 
experienced feelings. 


Computer breaks down 
mechanically due to 
data overload. 


Anthropomorphis of 
machine considered 
undesirable. 


11 


Parents nagged Jack 
about brushing his 
teeth; Jack expressed 
anger. 


Parents scold Jack for 
getting dirty; Jack 
expresses anger because 
parents jumped to 
conclvxsions . 


Cultural norms upset; 
situation needed in 
which Jack's anger is 
more justifiable. 

? 


. 15 


Central character meets 
a dragon. 


Dragon deleted. 


Too fantastic in otherw 
realistic episode. 


19 


Central characters 
rewarded by much 
attention (inclusion) . 


Central characters 
' have their say' 
(control) . 


Change necessitated by 
FIRO structure. 


28 * 


Central character 
assured that everyone 
liked her; no mention 
of her liking herself. 


Central character 
learns self respect 
and esteem.'' 


Self-esteem is an 
important concept of 
LliC program. 
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Tlevlslons to Program Characters 

Revisions to program characters include changes of sex, race or 
ethnicity, and behavioral characteristics. The nature and extent of 
such revisions were determined by the findings and recommendations des- 
cribed in the Affirmative Action Report (Section III of Chapter Five) 
and in the Consultants' Reviews (Section IV of Chapter Five), and were 
influenced by considerations of time, cost, and suitability in terms of 
the existing story line. 

Table 30 summarizes the quantitative revisions made to program 
characters. \ 

Table 31 compares the total numbers and percent of representation 
of human characters as presented in Table 5 (from the Affirmative Action 
Report) and Table 30, and percent representation called for in the RBS 
Guidelines. 

Although, quantitatively, representation by ethnicity in the RV 
does not equal the representation required by Affirmative Action, quali- 
tatively the revisions are considered appropriate and necessary. 

Tables 32 and 33 indicate revisions made to program characters. 
In both tables each character is described as portrayed in the FTV 
materials used in spring 1977, and as portrayed in the RV. Each of the 
headed columns should be self-explanatory with the possible exception 
of "visibility." 

Visibility is measured on two scales. One is quantitative, relat- 
ing to the number of illustrations of a given character, the number of 
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T.oxe 30 



Number of Chairacters Presented in the Revised Version 
of the Program, Broken Dovm by Sex and Ethnicity 





Sex^ 


Ethnicity^ 








Characters 


Total 
No. 


M 


F 




W 


B 


A 


H 


N-A 


? 


Main Characner& N 
Human 

(%) 


9 

( 5.8) 


(45.0) 


c 

(55.0) 




5 

(67.0) 


2 

(22.0) 




1 

(11.0) 






Main Characters N 
Non-Human 

(%) 


17 

(10.9) 


9 

(53.0) 


5 

(47.0) 














17 

(100.0) 


^Supporting Characters N 
Human 

(%) 


72 

(A6.5) 


42 

(58,0) 


30 

(42.0) 




47 

(65.2) 


15 

(20.8) 


3 

(4.1) 


5 

(6.9) 


2 

(3.0) 




Supporting Characters N 
Non-Human 

(%) 


( 2.5) 


3 

(75.0) 




1 

(25.0) 












£^ 

(100.0) 


*Activities Characters N 
Human 

(%) 


53 

(34.3) 


26 

(49.0) 


27 

(51.0) 




33 . 
(62.3) 


15 

(28.3) 


3 

(5.7 


2 

(3.7: 


• 




Total -J 
% of Total (%) 


155 

(100.0) 


84 

(54.2) 


70 

(45.2] 


1 

( .6) 


86 

(55.4) 


32 

(20.6) 


6 

(3.8: 


8 

(5.3 


2 

(1.2' 


21 

(13.7) 



1 Sex: M = male F = female 

2 Ethnicity: W « White B = Black A « Asian 

H » Hispanic origin N-A -Native American 

*The total numbers in these two. groups were reduced frop the field test 
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Table 31 



Comparison of Total Representation of Human Characters 
by Sex and Ethnicity for Two Versions of the Program Materials 
Compare!^ Against Balan.ce Required by Affirmative Action 







Sex* 


Ethnicity** 




Total N 


M 


F 


w • 


B 


A 


H 


N-A ■ 


? 


Affirmative 
Action requirement 


100 


50% 
(^10%) 


50% 
(^10%) 


75% 
(±10%) 


18% 
(±15%) 


2% 

(±1%) 


2% 
(±1%) 


2% 
(-1%) 




Field test 
version 


N 

181 

% 


104 
(57.5) 


77 
(42.5) 


80 

(44.2) 


37 

(20.4) 


0 
0 


0 

0 • 


0 
0 


64 
(35,0 


Revised 
version 


N 

134 

% 


72 
(53.7) 


62 ' 
(46.3) 


.86 

(64:2) 


32 . 
(23.9) 


6 

(4,5) 


8 

(5.9) 


2 

(1.5) 





*Sex: M = male F = female 

**Ethnicity: W = White B = Black A « Asian 

H = Hespanic origin N-A North American 
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Table 32 



Human Characters Represented In Two 
Versions of Program Materials 









Field Tpsjr 


V C L O XUil 




Revised 


Version 










DCiia V X U La X 






B^n^ivlo ral 


Characters 


V* 


Sex 


Ethnicity** 


\jlia Lak.LcLXoLXL. 


OCX 


CiLnnxcxLy^ 


Characteristic 


Jack 


VH 


M 


W 


active , Independent 


M 


W 


active. Independent 


King 


M 


M 


w 


c pn Q 1 Vi 1 p 


M 


W 


sensible 


Queen 


L 


F 


w 


UcpcllUcllL 


r 


W 


somewhat dependent 


D.Dlfferent 


M 


M 


w 


1 nHpnpnHpnt" 


p 


W 


Independent , 










adventurous 






aUVcIlLUruUa 


T,E, Thorn 








domineering. 






domineering. 


M. Mellow 


H 


F 


w 


H pnpTiH PTi f* 


r 


W 


somewhat dependent 


Zoo Keeper 


L 


M 


N 


hiiTTiH Tiitn 


M 


W 


humd r um 


Doctor 


L 


F 


w 


QflTTlPuhiSf' HpriPTinPTlf" 


p 


n 

D 


calm, sensible 


T.T.Howard 


L 


M 


w 


aL.Lxvcy cunxxuciiL 


M 


D 


active, donfldent 


Freddy F./ 


L 


M 


w 


tvnlral TV hn<l^ 


p 


u 
n 


Lypxcax iv nose. 


Juanlta J, 














Announcers 


VL 


M 


w 


nonentity 


M 


B 


one, not 2 char. 


Barker 


VL 


M 


w 


extrovert 


M 


H 


extrovert 


Guards 


VL 


M 


w 


military 


M 


H 


military • 


Driver 


VL 


M 


w ■ 


no dialogue 


M 


H 


no dialogue 


Herald 


VL 


M 


w . - 


clearly spoken 


M 


H 


clearly spoken 


HI Ho 


VL 


M 


N 


flowery, verbose 


M 


B 


articulate 


Freddy 


VL 


M 


w 


grinning 


M 


W 


smiling 


Vera 


VL 


F 




simpering soprano 


F 


W 


pleasant alto 


Larry 


VL 


M 


N 


fat, dirty, lazy 


M 


W 


not Illustrated 


Melvln 


VL 


M 


w 


bumbling' tough guy' 


M 


w 


bumbling' tough guy* 


Susie 


VL 


F 


w 


giggly, naive 


F 


w 


pleasant, cheerful 



* visibility: VH - very h4.gh ** Ethnicity: W - white 
H - high B - black 

M » medium H » Hispanic 

L " low No non-white 

VL ■■ very low 




story installments in which that character participates, and the amount 
of dialogue spoken^ The second scale of visibility is conceptual, re- 
lating to the strength of the impact a character has on the storyline, 
and/or the relative importance of a program concept which is taught • 
through the behavioral characteristics of the chairacter. A comparative 
example based on the field test version should illustrate the purpose 
of indicating visibility. ' . * ^ 

The Queen and Daringly Different are both main characte^rs. The 
former appears in 5 story installments, is illustrated in 11 of the 124* 
frames presented on the Storypages, and has many linesf of dialogue. 
However, although necessary to the storyline, she keeps a very low profile 
and plays a subordinate role. Quantitatively she has low to medium 
visibility; conceptually she has low visibility, resultfing in a combined 
rating of "low." By comparison. Daringly Different appears in 2 story 
installments, is illustrated dn 6 of the frames, and has less than half 
tlie. amount of dialogue given to the Queen. He carries total responsi- 
bility for portrayal of an important program concept, and. plays a posi- 
tive superordinate role. A combined rating of "medium" visibility re- 
suits for this character. ^ 

Table 32 reflates to human characters. All main characters, all 
titled but unnamed supporting characters, and named supporting characters 
specifically mentioned in either Section III or Section IV of Chapter 
Five are listed. Supporting -human characters and activities characters. 

•5 

not included in this table have been revised only in illustrations. 



196 

214 



Table 33 

Non-Human Characters 
Represented in .Two Versions of Program Materials 





Field 


Test Version 


Revised Version 


Characters 


Sex y 


Behavioral 


Char. 


Sex 


Behavioral Char. 


Asterisk 


M 


wise, professorial 


M 


wise, professorial 


dopser 


M • 


logical but 


cries 


M 


does not experience feelings 


A. Alligator 


M 


extrovert 




M 


extrovert 


B. Bluechip 


F 


somewhat dependent 


■ F 


somewhat dependent. 


Far Out^ Fish 


M 


abrupt 




M 


abrupt 


Lonely A Lot 


F 


as 


name 






as 


name 


Little Lonely. 


M 


as 


name 




r 


as 


name 


Mighty Glad 


M 


as 


name 




M 


as 


name 


Little Glad 


F 


as 


name 




F 


as 


name 


So Very Sad 


F 


as 


name 




F 


as 


name 


Little Sad 


F 


as 


name 




F 


as 


name 


Hopping Mad 


M 


as 


name 




M 


as 


name 


Little Mad 


M 


as 


name 




M 


as 


name 


Horribly Hurt 


F 


as 


name 




F 


as 


name 


Little Hurt 


F 


as 


name 




M 


as 


name 


Scared to Death 


F 


as 


name 




F 


as 


name 


Little Scared 


F 


as 


name 




F 


as 


name 
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Table 33 relates to non-human characters, all of whom are listed* 
Illustrations * Almost all illustrations of characters described 
as non-white in the Affirmative Action Report were redrawn. Many illus- 
trations of white children, especially girls, were also redrawn. When 
characters were revised'by changing sex or changing or identifying race 
or ethnicity, new illustrations were created. 

III. STUDENT MATERIALS 

This section ccjntains three parts: accessories, tape presentation, 
and Student Journals. Although the audio tapes are not actually handled 
by students, they are included in this section since students are direct 
ly effected by the content and 'nature of the tape presentation. 
Accessories . ' 

The FTV accessories are little changed in the RV. The mljior • 

changes that were made relate to cost-effective production. 

t> - * 

Tape Presentation 

There were several changes made to the tape" presentation of the 

Heartsmart stories and games. 

Four general revision strategies- were applied to the presentations 

of all the lessons in the program. These are discussed below. 

• The greeting, concluding statements, and standard directions given on 
the tapes for each lesson (e.g.., students were told during each lesson 
that they could color the Storypages) were found annoying and unneces- 
sary by both teachers and students after initial familiarization with 
the program sequence. Consequently, these statements and instructions 

. o » 
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were modified .or eliminated to reduce the amount of repetition. Les- 
son 9 is the last time students Are told' that they may color the 
Storypages. Lesson 8 is the last time students receive full clean up 
directions. 

• Since teachers and students found the frequent use of the words "now,** 
"well,** and "O.K." irritating, these words were almost entirely elim- 
inated from the tape presentation. Sentences beginning with **you 
should'* were also rephrased to sound less condescending. 

a Students were vehement in expressing dissatisfaction with the voice 
Intonations of the tape presentations, finding them too "babyish.** 
The actors and actresses who will ^dramatize the tape script will be 
cautionied to speak in a way which is more appropriate- to a fourth 
grade student. 

• The intervals allowed for the students to respond were at times too 
short or too long. Two remedies were adopted. The intervals on the 
tapes were either shortened or lengthened appropriately, or teachers 
were given the option to stop or speed up the tape as necessary. 

Othet changes made to the tape presentation Included': (1) remind- 
ing the students to supply their own answers on the Gamepages; and.. (2) 
all the modifications in directions which were necessary as a result of 
changes made to the Student Journals. 
Student Journals 

Major changes were made to the format of the FIV Student Journals. 

In some ways, the use of the word ''journal** by the developer to describe 

f 
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the student pages used during each lessorr-is slightly misleading and 
needs to be clarified, in order for the' reader .to undeistand the full 
impact of the revisions made. The FTV student "journals" consisted of 
unbound coloring, game and reviewpages which were distributed pjiecemeal. 
at the beginning of each lesson, by the teacher in Unit I and by the 
group leaders in Units II and III, There was no bound "journal" per se, 
^ The major format revision made to these journals was to combine 
and bind the loose . student pages into a booklet which the, students can 
keep during and. after the program. The major advartage of the RV stu- 
dent -booklets is that they greatly reduce the management tasks of the 
teacher, and accordingly enable smoother commencement*. of each lesson. 



A. 



Oth^er advantages of the RV booklets i^iclude'': •(!) reduction of bulk of 
the materials and hence reduction in shipping c<fsts and needed storage 
space; (-2) reduction of production costs; and (3) increased student 
motivation by allowing for individual ownership, 

A second major change made to the Student Journals (and to the 
program) was the deletion of the Reviewpages, and of the tape-led re- 
view which" occurred in the FTV at the end of e^ch lesson, (This revi- 
sion was discussed in the s*ection referring to the teacher's manual), ^ 
A f ew-revisions were made to the tape-led activities, (and related 
Gamepages)^ and of those made most related^- to format and layout of the 
Gamepages, Minor content changes were made on the tape-script and/or 
the Gamepages for Lessons 4, -8 and 9,^ ' 

One of the least popular tape-led activities was the one following 
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Lesson 15", a lesson focusing on "difference" in which' the- central 
character advises students not to be worried ox afraid of being differ- 
ent if they are true to themselves and don't hurt other people. The 
FTV tape- led activity asked students to express feelings about the 
central character and -to read each other's feelings. The P^V activity 
asks students to imagine a desirable "different" activity, career, or 
life style for themselves in thetr futures, and to discuss the differ- 
ences chosen- within their small groups. 

Accordingly to the FIRO structure. Lesson 19 relates to control. 
^However, the FTV story for Lesson 19 related to inclusion. Changes 
made to the story necessitated revisions to the tape-led activity for 
that lesson. In the RV tape- led activity, students role-play characters 
on a television show in which they express desires relating to control, 
instea^^^f des'ires relating to inclusion. 
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APPENDIX A 

PROGRAM IMPLEMENTATION AND PARENTAL CONSENT 

o 

Parental Consent Form 
Information Pages 
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XYZ SCHOOL DISTRICT 
ABC ELEMENTARY SCHOOL 



Dear Parent, 

Next term, your child's 4th grade class will be taking a course called The 
Heartsmart Adventures^ The course is described in the following pages. 

Because Heartsmart is new and different, we are asking you to decide 
whether or not your child will take the course. 

Please read the description carefully. 

If ,you have any questions now or at any time during the course, feel free 
to call: 

* Jane Roberts at Research for Better Schools 
i'"' 561-4100 (ext. 220) or 

Mr, Principal at ABC Elementary School 
(Phone number given) 



I have read the description of Heartsmart Adventures^ 
I agree/do not agree (Please circle one.) that my child 

(Name) may take the 

course called Heartsmart Adventures . 

I understand that if I give my consent now, I am free to withdraw my child 
from the course at any time. 



Signed_ 



(Parent or Guardian) 



Date 



JR:nls 
12-09-76 



^23 



"HEARTSMART ADVENTURES" 

"Heartsmart Adventures" has been developed over the last three 
years, with federal funds from the National Institute of Education, by 
Betty Berzon, a psychologist , and staff members of Research for Better Schools. 

Research for Better Schools is in Philadelphia, and is one of 
several educational research and development institution in the U.S. 

. "Heartsmart Adventures" has been tried out in schools in Los Angeles. 
It is now being tried out in the Philadelphia area.' Next year it will be 
offered for commercial publication. 
Descriptive Overview 

The "Heartsmart Adventures" course consists of 30 lessons, each 
designed to take about 2S minutes. Twenty-four of the lessons are based 
on dramatized tapes. The remaining 6 lessons are discussions led by the 
teacher. 

The tapes tell the dramatized story of a boy named Jack who has 
many adventures as he finds his feelings and learns to express them. In 
the story. Jack learns, among other things, to understand other people 
and himself better. 

Jhe tapes also guide students in activities for which they use 
game-pages and/or other materials. (For example, in Lesson 9, each 
student looks at a picture while the teacher on the cape explains that 
the picture shows Joe pushing Billy. Each student is then asked to 
choose the feeling Billy might have, and underline the feeling on the 
list at the side of the picture. Four different situations are described 
and illustrated and students are asked, to identify the most probable 
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feeling of the main character in each situation. The teacher on the tape 
then reviews each situation explaining that the ''answers" given are the 
most likely for most people.) 

The course is divided into three units. In Unit I (the first 12 
lessons) , students work individually. For the rest of the course 
(lessons 13-30), students work in groups of six. 
Detailed Outline 

Below are brief descriptions of the 30 lessons in the course. An 

asterisk (-) is used to indicate any activity that students may find 

unusual, new or different. 

' 1. Senses — Jack goes into the world to find what he is missJ.ng. 
Students relate senses to experience. 

2. Feelings — Jack meets the Feelings. 

Students relate the feelings "sad" and "happy" to experience. 

3. Verbal expression — Jack learns to speak his feelings. 

Students respond to taped stimuli by stating their own feelings 

4. Listening — Jack learns to listen to the feelings of others. 

Students identify feelings expressed by tones of voice on 
tape. 

5. "Reading" — Jack learns to "read" feelings of others expressed 

by facial expressions or pantomine. 
Students "read" illustrated feelings. 

6. Heart to Heart — The teacher leads the class in a discussion 

reviewing previous lessons, and encouraging students to 
relate concepts to personal experience. 

7. Variety in expression — Jack competes in '*l've Got A Feeling." 

Students "read" feelings expressed by volunteers who respond 
to directions given by the teacher. 

8. How not to make mistakes — Jack and a robot called OOPSER 

(Only Original Person Scanner) help others to see Tiow and 
why they may have "misread" others' feellnes. 
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Students evaluate taped situations, identifying feelings, 
dramatized and ways in which to check out what others feel. 

9. Cause and effect — Jack and the OOPSER help others to see how 
feelings can cause actions (behavior), and vice versa. 
Given a situation (taped or illustrated), students name the 
feeling most likely to be felt by the main character in the 
situation. Students respond in the same way to four different 
situations. 

10. Cooperation — On his way home to the palace. Jack learns to 

cooperate. 

Students work in groups on a class coloring project. 

11. Transfer — Jack learns that he can transfer what he has learned 

"out in the world" to situations at home. 

Students match learned concepts to a variety of places and 

situations outside school. 

12. Heart-to-Heart Talk 

.. 

13. How not to be lonely — Jack helps the Droop, a lonely fantasy 

character, to form a group\to meet and make friends. 
Students identify their color-coded groups and then discrimi- 
nate between taped examples of groups sharing feelings and 
groups not sharing feelings. 

" 14. 'Inclusion ~ Jack and the Feelings decide to set up a Heartsmart 
School in the Palace. 

In each group, students role play a T.V. show, selecting 
1 characters from a given list of three possibilities, first 
reading given responses., and then improvizing. 

15. Control — Daringly Different describes his adventures and 

explains that each, person is different in his/her own way. 
Students imagine that Daringly Different is coming to visit 
them, and show their feelings about this visit by facial 
expression. Each student "reads" the feelings of other 
group members. Discussion follows. 

0 

16. - Affection — On a trip to the zoo, the. Feelings decide to break 

their usual partnerships and make friends with others as well. 
The Heartstart students play a game of pairing zoo animals in 
unusual ways. 

*In each group, each student chooses two other students and 
describes, in writing, an activity they might enjoy Sharing. 
Group discussion follows. 

17. Heart-to-Heart Talk 
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18. Inclusion ~ Two of the Feelings run away because they are 

being ignored. 

Students identify their own feelings stimulated by a story. 
They then recall occasions when they felt left out, and identify 
their feelings at that time. 

19. Control — The Feelings who ran away return to the palace where 

they proclaim a "Pay Attention To Me Day". 

Students role play in a T.V. show in which they state- ways in 
which they would like to receive more attention. 

20. Affection — Jack and the Feelings learn how to make friends. 

*Each student sends a "tell-a-gram" to two other group members 
asking to make friends and completing the sentence, "Something 
you do ^hat I really like is ..... "" 

21. Heart-to-Heart Talk 

22. Inclusion — The Feelings learn that everyone needs to feel 

important . ^ 

Students role play "Very Important People" in a radio show, 
giving reasons why they are important. 

23. Control ~ The Royal Reading Feelings Championship takes 

place at the palace. All the characters take part. 
Students play a version of the Royal Reading Feelings game. 
Each student then states what he/she would like to do especial- 
■ ly well. 

24. Affection — The students of Jack's Heartsmart School hold a 

graduation dance, where the King, Jack's father, finds out 
that he is likeable as a person. 
*Each student sends a tell-a-gram to two other group members 
saying, "I want you to like me." Answer-^grams giving 
positive friendly responses are returned. 

25. Heart to Heart Talk 

26. Inclusion — The Feelings get pushed around by a T.V. director — 

T. E. Thorn — and decide that before they can work with 

Ms. Thorn they'll have to hold a Heart to Heart talk. 

Each student recalls an occasion when he/she felt unimportant 

and states the feeling shown "on the outside." 

Each student states the feeling felt ''on • the inside". Group 

discussion follows. 
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27. Control — The Feelings hold a Heart-to-Heart talk with 

T» E. Thorn, who reveals some of her feelings. 

Each group cooperates in coloring a large picture, changing 

the sketch from a gloomy scene to a bright one. 

28. Affection — T. E; Thorn suffers from the "Like-mes" — a 

confusion of expressed feelings, and is cured when she learns 
♦ that she will still be liked if she is a little more self- 

controlled. 

*Each student writes a tell-a-gram to each group member stating 
appropriate reasons why the receiver is liked. Answer-grams 
are returned. Discussions follow. 

29. Heart to Heart Talk ' ^ 

30. Conclusion ~ All the characters hold a "Hello and Gdodby" 

party. 

Students draw pictures of anytMng enjoyed during the 
program. 

Discussion follows. * , . 

Ways In Which Children May Benefit from "Heartsma rt Adventures" 
The course is designed to help children to: 
. recognize their own feelings; 

. express their feelings in words, actions , 'or^ facial expressions; 
. understand that everyone has feelings; 

"read" other people's feelings by listening carefully to what 

others say or watching how others behave; 
. understand that people often behave in certain ways because of 

the feelings they have; ^ . 

. identify ways^in which people may behave when they have certain 
. feelings^ ' ' 

identify feelings pepple may have that cause them to behave in 

certain ways; 
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be able to -express (by words or actions) what they want or 
don't want (from other children aod/or adults) in the areas of 
affection, being included in .activities, or being in charge; 
During the course, children will be involved in reading, writing, 

class and group discussions, brief dramatizations, and projects led by 

the teacher. . . 

Ways In Which Children May Feel Discomforted By "Heartsmar t Adventures" 
During discussions children are asked to describe their feelings 

or recall situations in which they experienced certad^feeU^^ 

There are two kinds of discussions- 

On six occasions, the teacher leads a class discussion. Students 
are told that "if they want to" they may describe their feelings and 
behavior and 'the situation they experienced.^ If students choose to do 
this, they may name other people. This may cause some embarrassment to 
people named, and the student speaking may feel embarrased by the reaction 

of others in the class. 

In, ten of the lessons in which students work-, in groups, the last 
3 to 5 minutes of the lesson are spent in group discussion.' The teacher 
on the tape directs students.,to think about the activity jus.t completed 
and to tell other members of' the group their feelings about the activity. 
During this discuTsion, students may cause each other embarrassment if 
negative comments about others' behavior are made. 

The activities following lessons 16, 20, and 24 are based on 
"tell-a-grams". Each student is asked to express friendship toward other 
group members and to describe a likeable behavior of those to whom he or 




she sends a tell-a-gram. In'ciach of these -three activities all student^s 
will receive at least one tell-a-gram. Some students may receive two or 
more. It is possible that students who receive only one tell-a-gram will 
experience a feeling of being left cut. 
Alternative Programs or Procedures 

Although a variety of affective programs and activities are used 
in elementary schools, as far as is known, there is no prograk^ currently 
in use which has the same goals or is designed to achieve the same results 
as "Heartsmart Adventures." 




/ 
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^ "heartsmart Adventures" Story Summary 

This is a story about the J^ck-of.-Hearts , who lives in the Royal 
Palace with his mother, the Queen-of-Heartc. and his father., the King-of- 
Hearts. The palace is a beautiful place, but Jack is not happy there. 

Som'Sthing is missing in. his life, but he doesn't know what it is, so he 

^ i . . • 

decides to leave home, and go into the world to see what he is missing. ■ 

• His adventures begin at once. In the Puzzles Galore Park, he meets 
a Far-Oi).t-Fish who helps him come to his senses. That is. di>3covers 
that he has senses that' provide useful information about what is happen- 
ing to him. 

In his next adventure. Jack meets the Wise Old Asterisk, who helps 
people find the information they need when they have a special problem. 
The Wise Old Asterisk discovers that Jack doesn't know what feelings are, 
so he introduces him to' a few. There's Mighty Glad and A Little Glad, 
Scared to Death and A Little ' Scared , Hopping Mad and A Little Mad, 
Lonely A Lot and A Little Lonely, Horribly Hurt and A Little. Hurt , So 
Very Sad and A Little Sad. 

These Feelings become Jack's friends," and with their help he be- 
comes aware that he has feelings of his own. He learns to express his ■ 
feelings in different' ways , with, and without words. He becomes more 
aware of the feelings of others, and how these feelings affect the way 
' others behave toward' hiT.., and how that affects how he feels, about them 
and behaves toward them. This helps him understand how to work cooper- 
atively with others to get things done that can't be accomplished by 
people working alone. 
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He learns to "read" the non-verbal language of emotions (facial 
expressions, tone of vdic4, etc .;) add he learns the importance of ^ check- 
^ing out other people's meanings when their messages are unclear. \He 
becomes so proficient at all this that he is soon considered an expert, 

and in this^role, he co-hosts a television program called "Ask the , 

•' ■■ /•>' * 

OOPSER," on which people appear to get help with their perplexing people 
puzzles. The OOPSER is a computer (The Only Original Person Scanner) 
who signals interpersonal lapses and oversights when people are telling 
their stories by sounding out OOPS! OOPS! OOPS! OOPS! OOPS! ■ 

Of course, with all this learning about feelings and relationships, 
'Jack's heretofore-missing heart is now very much a part of him, and 
growing stronger everyday. 

At the end of- Unit I, Jack becomes homesick for his parents, and he- 
returns to the Royal Palace with his new fri<€nds, the Feelings and the 
Wise Old' Asterisk. His parents are very glad to see him and to learn 
that he's found his heart. Soon Jack realizes that the lessons he 
learned out in the world about feelings and relationships apply > just -as 
much at hpme, with his parents. That makes all the Hearts very happy. 

Back home, in the Kingdom of the Hearts, Jack discovers tHkt there 
are others who have not yet found their hearts. With the help pf his 
father, the King, he begins the Hearty Start School where he and his 
friends ''teach the lessons of the heart that he learned in his adventures 
in the world. At the school, the students learn about making friends, 
about cooperation and sharing leadership, asking for the attencion they 
need, expressing their individuality, being aware of the feelings* of 
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others, being supportive and responsive to others • 

They learn that it's all right to want to, belong to a group, or not 
to want toC They learn that everyone wants to be important to others, 
that people like to be in charge sometimes, but not other times, and 
some people don't like to be in charge at all, which is just fine. They 
learn that it's O.K. to want to be good'^at some things, and to want to 
be liked by others. And, they learn how to let people know what they want 
in ways that are likely to have successful outcomes. 

Some very interesting people come to the Heart Start School, like 
Fast-Talking Howard of the Royal Television Network, who comes to broad- 
cast the, opening of the school, and the Royal Reading Feelings Champion- 
ship Gaines that are held during graduation weekend between the Heart 
Start Students and the Royal Heart Marines. 

There's the world-famous adventurer, Daringly Different, who tells 
about learning that it was O.K. to be/different from others, and feeling 
so strong as a result that he began -to do the daring things that made 
him world-famous j 

Unusual things happen, like /the time the Hurts go underground 
because they aren't getting enciugh attention. Down, down, down they 
they go to the subterranean passages of the Royal Palace where they nieet 
the Altogether Alligator, whg^ takes them to a meeting of the Speak Easy 
CluU, where you can learn to speak easy about yourself. There they hear 
Ms. Bessie Bluechip sing a song: '*If you're feeling left out, don't just 
sit and stew. Tell tne what you want, c'mon speak up about youT' That is 
an important learning experience for the Hurts who go back up above 
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ground to get the King to proclaim a "Pay Attention to Me Day," on which 

f ^ 

the Hurts pass on Ms; Bluechip's message to every one in the Kingdom of 
the Hearts. 

Then there is graduation weekend, with the Royal Reading Feelings 
Games and the Crystal Ball,- where the Queen gives out the Glass Heart 
Awards to the Heart Snart Students who have learned best how to express 
''themselves in different ways. 

Now the school has become so famous that the Barbarian.. Broadcasting 
Company has sent their best director, Ms. Thistle E. Thorn, with'a crew, 
to film a television special on the Heart Start School. The great 
excitement about that is dampened by Thistle £.'s very prickly manner. 
She pushes the Little Feelings out of the way ("It's big news we're 
after!") and miscasts the Big Feelings, until everyone is angry with her. 
In line with the policy of the school, they call a Heart-to-Heart Talk- 
with her to let her know how they feel about what is happening. To show 
her that they know best what they can do, they put on a show for her^ in 
which they confront her with herself as they see her. This affects 
Thistle £. , who breaks down and tells them her real name is Marsha Mellow 
and that she is really a softie inside. They like her much better then, 
and all cooperate to put on a terrific television special. 

At the end of the school year, the Heart Starters have a hello and ' 

goodbye partyi gppdhye because they're leaving each other, hello 

because they're going to do new things and meet new people. Jack sings 

a song at the party, which ends with 

So follow your own path, 
and follow it true. 
You'll find something super; 
that someone is you . 
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1. Teacher Training Session Ap;enda 

2. Teacher Training Session Evaluation Results 
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TEACHER TRAINING SESSION 
AGENDA 

December 16,1976 ^ 
9 a.m. — 4 a.m. 

Introduction . Participants introduce themselves. 

Program Rationale . Developer (Betty Berzon) introduces concepts; participants 
role play program activities related to concepts. Developer explains and 
illustrates goals by referring to the Teacher's Manual. 

Program Development History . Developer explains. 

Program Organization and Materials . Developer explains; participants refer ^ 
to relevant sections of the Teacher's Manual and to materials. 

Lesson 14 . Participants form groups and role play \the tape-led activity for 
Lesson 14, Discussion follows; 

Lunch . / 

Group Formation and Leadership . Developer explains; participants refer to 
the Teacher' r Manual. 

Trouble Shooting . Participants refer to the Teacher's Manual; developer 
answers questions about potential problems. 

Heart to Heart Talks . Developer explains and involves participants in role 
playing a Heart to Heart talk. 

General Guidelines > Developer makes specific suggestions relating to program 
implementation and answers questions posed by participants. 

Training Session Evaluation . Participants respond to evaluation questionnaire. 
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HEARTSMART TRAINING IN RETROSPECT 



Directions: 

Follow the individual directions given for each question. Feel free to add 
your own comments on the back. 



1. Did the training provide you with a good understanding of the Heartsmaist ' 

Indicate whether the training you received on each of the following topics 
was or was not sufficient, (check one for each topics) 

Topics ' insufficient sufflcieut 

a. Goals and concepts of Heart smart 11 

b. Content and organization of the 

program 11 



d. Procedure for organizing small groups ^ 
and leading small group discussions ^ 



e. Procedure for Heart to Heart talks 

if. Anticipating and dealing with 
potential problems 



see attached 



If yes, please specify and explain_ 



c. Procedure for conducting the lessons / n 



10 



11 



Are there any topics or experiences that 
should be added to the training program? 

(circle one) YES ^ NO 5 
If yes, please specify and explain ■ 



3. Are there any L pics or experiences that ■ * 
should be "eliminated, f rem ', the^- training 

program? (circle one) YES 1 NO 



4. Are there any topics or experiences that 
should receive more emphasis? 

(circle one) YES 2 NO 7 

If yes, please list (the topics) using TOPICS OR EXPERIENCES TO BE 

the key letters from question #1. EMPHASIZED 

1) 

2) 

see attched 

'"/ 3) 
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5. How well did the training piopare you to deal with specific problems? 
Indicate how confident you feel in your ability to handle the following 
problems should they arise, (circle one^ each group^ 



Problems 



not'"^ fairly 
confident confident confident 



a* 



students being noisy or disruptive 1 2 3^'^^ 4^^^ 5^ 

b. unusual behavior (s) of students 1 ^ ^^^^ 4C^^ 5^ ^ 

c. lack of motivation of students 1 ? 3^2) ^O) 5^) 



d. difficulty with tape because of 12 3(1) 4(5) 5(5)\ 
readability level 

e. peer rejection or criticism 1 2 3(^) 4(6) sd) 
among students 

f. potentially embarrassing situations/ 1 2 3(2) 4(8) sd) 
discussions (for Ss) 

g. negative comments from parents 1 2 3C3) 4(41 5(3) 

h. personal uneasiness, if any, with 1 2 3(2} 4(4) 5(5) 
Heartsmart concepts (on your part) 

Did you have enough opportunity to practice your roles in implementing the 

Heartsmart Program? {circle one) YES / NO J 

If no, please explain see attached _ 



7. How helpful did you find the different parts of the training? 





not 
helpful 


fairly 
helpful 


very 
helpful 


discussing background and development 


1 


2 3(27 


4(2) 


5a)- 


listening to tape/completing game pages 


1 


2 3(1) 


4(1) 


5(9) 


discussing problem areas 


.1 


2 3(3) 


4(1) 


5(7) 


role-playing for Heart to Heart talks 


1 


2 3(4) 


4(1) 


5(5) 


role-playing for parental concerns 


1 


2(2;) 3(1) 


4(3) 


5(5) 


teacher quiz 

what if . . . sequence 


1 

' 1 


2 3 
2 3 


4 
4 


5 no 
5 

tr; 
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8. How confident to you feel about your role for each of the following? 
(oirale one) " • 

not fairly 
confident confident confident 

story activities 1 2 3 QX 4 (IX 5 (grj 

tape~led excercises 12 3 4 (3) 5 (8) 

Heart to Heart talks 1 (1) 2 3 (2) 4 (2) 5 (6) 

optional activities 1 (1) 2 3 (1) 4 (2) 5 (7) 

9. How much did your interest in .Heartsmart increase or decrease as a result 
of the training sessions? (circle one) 

1 2 (.1) 3 (5) 4 (1) 5 C4) 

decreased remained increased 

a great the same a great 

deal ■ * deal 

10. I^Jhat was most and least interesting and/or enjoyable to you about the Heart- 
smart training session? 

- see attached 
Most: ■ 



Least: Sjee^ attac^tgd 



11. Was enough time devoted to answering your particular questions, i,e^ 
addressing your particular concerns? 

YES 11 

NO (please explain) 



see attached 



239 



12. If it were optional, would you attend this type of training in the future? 
YES 10 

NO (please explain) 

see attached 



\ 



13. Any further comments? 

On the Heart smart program: - 



i 


see attached 


• \ 




■ • ■ • . ' \ ■ 


On the Heartsmari 


: training session: 




\ 

\ ■ . 




■ \ 
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HEARTSMART TRAINING IN RETROSPECT 
SUMMARY OF COMMENTS 



B2 



2. Are there any topics or experiences that should be added to the 
training program? 

YES, but I would need more time to think about specifics* 
.Maybe a time line — when to start and rough dates for lessons • 



Seeing. a film or slider of an actual lesson would have been 
beneficial* 

Film or filmstrip or tape of groups in action. Ongoing training 
sessions after completing a portion or portions of thfe program* 

Added meeting later to interact with these same workshop people 
regarding experiences encountered. - 

3. Are there any topics or experiences that should be eliminated from 
the training program? 

Can a child deal with a heart-to-heart talk" that bares his failures 
without offering a solution? 

Introduce each other. Role, playing. 

4. Are there any topics or experiences that should receive more emphasis? 

li Conceptual Framewor:k, 2) Some background reading for teachers, 
3) A training program for the admin, of the schools where used. 

1) More positive experiences and awarenesses. 

6* Did you have' enough opportunity to practice your roles in implementing 
the Heartsmart Program? 

Needed more workshop . type activities like the "TV" activity to 
practice teacher directed lessons. 

More hands on activities. 

Not possible to conduct an actual lesson. 



More activity type experiences rather than lecture on manual. 




COMMENTS com . 

10. Wliat was most and least interesting and/or enjoyable to you abN?ut 
^ the lleartsmart training session? \ , 

Most:. Knowing something about the theoretical base upon which thXs 
program is built. 

Most: Discussion of actual course of study. 

Most: Practical- application, sharing ideas, being aware of problems, 
supportive attitude present. 

Most: Hands on activities. 

Most: Participating in the activity. 

Most: The teacher discussions were most interesting. Also, hearing 
precise information about program was indeed helpful. 

Most: Going thru a lesson. Learning objectives. 

Most: Actual work with tape and materials. 

Most: Activities where we were involved. 

Most: Actually using the materials and hearing the tapes. 
Most: Role playing a taped lesson. 
Most: Theoretical background. 

Least: "Quickie" walkthrough of teach,er-guide'. 

Least: Too much lecture. * 

Least: When we had to listen to wh^t we've already read in the book. 

Least: The day was much too long for me. Two-thirty was a good 
/ ■ time to stop. 

Least: Role playing projected problems. 

Least: Session dragged. More involvement needed. Could have been 
done in less time. 

Least: Length of time of the day (this may not be changeable but I 
found the co.lor of the walls in the room very distracting 
and irritating to sit in for a long period of time). 

Least: Specific situations/problems. H~H simulation. 

12. If it were optional, would you attend this type of training in the 
future? 

YES, but only if more concise. 

YES, if I was involved in piloting a program or in the study like 
this time. 

YES, assuming the program was effective based on field test 
experiences and results. 
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COtlMKNTS CONT , 

13 • Any further comments? 

On the Heartsmart program: 

As the evaiuator, I'm still unclear as to the overriding goal. of 
the program. 

Good idea but I seriously ' doubt very much If .our district would 
ever put out the money to buy itl 

Seems to be an area of need of our youth— am anxious to begin, 
glad that resource people will be available fci problems, 
suggestions, etc. 

Looking forward to seeing if this program can make children 
more aware of themselves & their emotions • 

I'd like to meet again when the program is well underway, I'd 
like to talk with the other teachers involved • 

The goals of program are important. 

Booklet well-organized and clear • 

I am anxious to get more involved with the program and its 
activities. 

On the Heartsmart training session: 

I think the presenter was very knowledgeable but seemed to cover 
a lot too quickly. 

Instead of just lecture, perhaps a film, filmstrip and/or slides 
could be used to show examples of the group discussions, Heart-to-^c? 
heart. talks, etc. Have another session for feedback and results. 

Too long for what we did. 

It was too long. 

Maybe two morning sessions instead of one day— teachers tend to 
"droop" in the afternoon.' 

Enjoyable but long day. » 

Good— sharing session, cleared up many points — my questions were 
answered, got to know people related to program — stimulating. 
Would find it most beneficial and interesting if we could all meet 
again at the end of the program to discuss results. 

I think it would be beneficial to you and ourselves to meet again 
at tlie completion of the program. 

Too long— but sufficiently interesting to maintain my attention.. 
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EVALUATION MEASURES 



!• a!) Teacher's Opinion Surveys and Records - Directions 
Teacher Data 

Course Schedule . ' 

i)\ Form 1 - Lessons 
e)\Fonn 2 - Heart to Heart Talks 
f^Form 3 - Program Units 
gMForm 4 - Total Course 

2, a) ^udent Attendance - Directions 
student. Attendance - Record Sheet 

3, a) Studei^t Opinion Surveys - Directions 

b) ' Survey A - Tape- led Lessons 

c) Sufvey B - Heart to Heart Talks 

d) Survey C - Groups 

4, a) Observation forms and questions for Tape-Led Lessons 
. b) Obs^vation questions for Heart to Heart Talks, 

5, Student \Interview form, 

6, Teacher Interview form, 

*i 

7, a) What Would You Do? Test 

b) What Would jfou Do? - Scoring Key . 

c) What's Happening? Test 

d) N What's Happening? - Scoring Key 

8, a) My Class Inventory (3 scales) Measure 

b) FIRO-BC (modified from published version) Measure 



\ 
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Teacher Opinion Surveys 
and Recor(3s 



Teacher Data page. 



Course Schedule. 



Please respond BEFORE reading the Teacher's Manual. 

V 

Please keep this record up-to-date. 



There are 4 different survey forms, all designed to collect Information so 
that the developers of this course will be able tq usecyour experience and 
judgment in determining desirable revisions. 

Please respond to the appropriate survey as soon as possible after teaching 
a lesson or unii» ^ ^ ^ 

— T 

Form 1. There are 24 copies of this survey form, one to be compJ.eted after 

each lesson . 

Form 2. There are 6 copies of this survey form, one -to be completed after 

each heart-to-heart talk. 

Form 3. There are 3 copies of this survey form, one to be completed after 

each unit . 

Form 4. There is 1 copy of this survey form, to be completed at the end of 

the course. 



* 7.f you choose a starred C*) response to any question, pl^ease explain. 
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!j Teacher Data * * 

e 

Please respond to this questionnaire before you study the course materials. 
A. In this ^school: \ 

\ \ 

1. I teach grades ^ 



£• The average number of student^ per class that I teaches 

3. The main subject I teach .is ' ■ 

4. Other subjects I teach are ^ ■ 



B. Background: 

5, My major(s) in college' (other than education) 



6. I have taught for years. 

7. I have taught grades 



8, Most of my experience has been with . grade(sO, in the subject 

ar^as ; 

C. General: 

9. I wish I could educate children (so they could ...•-) 

/ a. . ' 

b. 



2i6 



Course Schedule 



As soon as possible after each ;lesson, please complete this lo^. 



Lesson 


Date 


Optional Activity (Give 
number Cs) of activity 
used J or x^ite "none" 
or "my own") 


Date optional 
ac uivii-y uaugiii. ^ 
if different 
from lessdn date 


Time taken 
for optional 

uji. vj.i.jr 


1 






r 




2 






■\ 




3 




















5 










6 










7 










o 
O 








— t 


9 










1 A 

10 










11 










12 






J 




13 










14 ^ 










15 








/ 


16 










17 










18 










ly 










2u 










01 




















/J 










24 










25 










26, 










27 










28 - - 










29 










30 
















f 





Le&son // 



Teacher ^ s Opinion Survey Form //I — Lessons 
Civole the appropriate number or responseis) and comment where neoessa/^y. 
1. How difficult were the materials to distribuLe and collect? (Ciy^.le o.ie. ) 



very difficult easy very 

difficult easy 

2. How v.uuld you rat':i the taped story for this lesson with respect to ^,he 
following? (Circle one for each quality*) , 




Qualities 


Very poor 


Poor 


So-So Good 


Very I 


a. volume (loudness) ^ 


1 


2 


3 


4 


5 


b. pace/ speed 


1 


2 


3 . 


4 


5 


c. conceptual clarity 


1 




3 


4 


5 


d- appeal of characters 


1 


2 


3 


4 


5 


e. appeal of plot/storyline 


1 


2 


3 


4 


5 


f. overall quality 


1 


2 


3 


4 


5 


How would you evaluate the directions given for 
(Circle one.) 


the taped 


.game 


? 


1 . 2 


3 




4 






V^^y confusing 
(Confusing 


clear 




very, 
clear 






How important are the concepts 


of today's 


lesson 


for your 


students? 



(Circle one. ) 



^not a little fairly very , extremely 

^important important important important important 

l\ 

\ , • . ■ - \ 

5. Was today's taped game relevant to the lesson's objectives? (Circle one. ) 

Yes No \ 



/ 

/ 



Did you use an optional activity ar given in the teacher's manual? 
(Circle one . ) - 

Yes No 

IF YES, ANSWER QUESTIONS 6a • AND 6b • IF NO, GO ON TO QUESTION 7.. 

a- Why did you choose to use it? (Circle all that apply.) 

1 liked I thought It reinforced It reinforced *Other 

it. Ss would lesson objectives. course 

like it. objectives. 



*Please explain 



b. Was it easy to teach? (Circle one . ) 

Yes *No 



*Please explain 

ANSWER THIS QUESTION ONLY IF YOU ANSWERED "NO" FOR QUESTION 6. 

Why did you choose not to use an optional activity? (Circle all that 
apply.) 

No time Unappealing Did not *Made up *Other 

available to me and/or reinforce my own 
to students objectives 



*P lease explain 



If anything out of the. ordinary (good or bad) happened during today's 
lesson, please describe. Also, feel free to make any suggestions for 
improvemant, or conpents\relevant to today's lesson. 



2iu 



Lesson // 

Time taken 

Teacher's Opinion Survey Form #2 — Heart-to-Heart Talks 

Circle the appropriate number or response and Qomrnent where necessary. 

1. How easy was it for you to present the "main points" and "questions" for 
today's lesson? (Circle one. ) 

. *difficult *fairly fairly easy 

difficult easy 

*Please explain 

2. How did you feel about the concepts of the Heart-to-HearttTalk? 
(Circle one. ) 

Unenthus-iastic t. Enthusiastic 

3. How much feedback did you offer students? (Circle one * ) 

very little a little some quite a lot 

4a. Approximately what percent of the class responded to your questions? 
(Circle one. ) . 

0-20% 21-41% . 41-60% 61-80% 81-100% 

u 

b. To what do you attribute this amount of response? 



5. In general, how (with what attitude) did your students offer responses? 
(Circle one. ) 



reluctantly fairly eagerly *other 

willingly 

^Please explain . 



6. How well did the students attend to each other's comments? (Circle one*) 



poorly not very well very *other 
well well 

'^Please explain 
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7. Was there much negative peer reaction (e.g. teasing, taunting, laughing 
inappropriately)? (Cirale one. ) 

*Yes No 

^Please explain ^ 



8. What was your overall opinion of today's Heart- to-Heart Talk? 
(Cirale one* ) 

*Not worthwhile fairly worthwhile very worthwhile 
*Please explain 



9. Please describe anything out of the ordinary that happened during today's 
lesson, and/or to make additional comments or suggestions. 



\ 
\ 



\ 
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Unit 

Teacher's Opinion Survey Form #3 — Units 

Cirole the appropriate response (s) and/or comment. \ 



la. Excluding the Heart-to-Heart Talks, the 2 best story installments were 
(in order cf preference) : 

(1) Lesson // (2) Lesson # 



I selected the installment ranked first because: (Cirole all that 
apply.) 

it was the content the story/ S's reactions *other 

clear and seemed relevant/ plot was were positive 
easy to important enj oyable 

understand 



*Please explain 



2a* Excluding the Heart-to-Heart Talks, the 2 worst story installments iriote 
the woi*st first) were: j ' 

(1) Lesson # (2) Lesson # / 

b. I selected the installment ranked worst because: (Circle all that 
apply.) ' 

it was the content the story/ S's reactions *other 

difficult seemed plot was were negative 

to understand irrelevant not enjoyable 

*Please explain 

3* The tape-led game I liked best related to gamepage # following 

Lesson if because it: (Circle all that apply.) 

appealed was well reinforced required Ss *other 
to the ' timed course to apply 

Ss objectives desirable 

skills 

*^Please explain 
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4. The tape-led game I liked least related to gamepage //^ 
Leisson # because it: (Circle all that apply.) 



did not was badly did not 

appeal to timed reinforce 
Ss course 

objectives 



following 



required skills *other 
Ss do not have 



*Please explain 



5a. The most successful optional activity that I taught was: (Fill in and 
circle one. ) 



following Lesson // 



my own following Lesson # . 
b. It was successful Kecause: (Circle all that apply.) 



it motivated 
Ss well 



it" clarified 
concepts 



it was the 
most creative 



it was 
easy to 
teach 



Mother 



*Please explain 



6: Please comment on any part of this unit in any way, you wish. You may want 
to consider: timing, administration, student materials, teacher's manual, 
etc. 



: ■ • • ■ \\ 

253 



cig 



\ ■ . 

Teacher's Opinion Survey Form //A — Total Course 

Cirole one number for each component for each dimension. 
a. Enjoyment of the students 



7 



very a 
little little 



.story installments 
. tape-led games 
•Heart-to-Heart Talks . 
.Optional activities 

b. Ease of administration 



1 , 
1 
1 
1 



very 

difficult 



2 
2 
2 
2 



some 

3 
3 
3 
3 



quite 
a lot 

4 
4 
4 
4 



•story installments 1 

. tape-led games 1 

.Heart-to^Heart Talks- 1 

.Optional activities 1 

Relevance to course objectives 



difficult 

2 
2 
2 
2 



easy 

3 
3 
3 
3 



very a 
I'ittle little 



some 



quite 
a lot 



a 

great 
deal 

5 
5 
5 
5 



very 
easy 

4 
4 
4 
4 



a 

great 
deal 



• Story Installments 1 2 3 . 4 5 

. tape-led games 1 2 3 4*5 

.Keart-to-Heart Talks > 1 2 3 4 5 

.Optional activities 1 2 3 4 5 

Comment bi^iefly for eaoh question. 

a. How Importani: are the course goals (see pages 1-2 in the teacher's 
manual) to elementary education? 



b. How important are the course goals to perso^nal development 
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c. In general, how well did the course achieve its goals? 



d. In what ways, if any, do you think the course produced changes in 
out-of-school behavior? i ■ 



e. Describe any examples you know of students using Heartsmart Program- 
related ideas, vocabulary or activities on their own: 



Describe anything unexpected that has happened with any of your 
students recently that you think might be related to the Heartsmart 
Program: 



Wi\ite in the information requested and/or circle the appropriate response.. 

a. If this course were to be commercially published, I would hope that 
it be: (Circle one) | 

used only with 4th graders 

used with grade(s) (indicate grade(s)) 

used only witl? those Ss who (please describe) 



b. If I were asked to teach this cours.e_.again,--m^ negative 
reaction would b e ^ \ ' 



c. If I were asked to teach this course again, my strongest negative 
reaction would be . 



d. I would/would not (Circle one ) recommend this course to my colleagues 
because 



Civol^ the appropriate response and comment where necessary. 
a. How difficult were the materials to: 





very 






very 




difficult 


difficult 


easy 


easy 


prepare 


^ 1 


2 


3 


4 


collect/put away 


1 


2 


3 


4 


store 


1 


2 


3 


4 


In comparison to 


other courses, 


does '*Heartsmart 


require more 


teacher 



preparation time? Please do not consider field test record keeping, 
etc. , as preparation. ( Circle one ) 

*Yes No 

*Please explain 

c. Did you encounter many procedural difficulties with ''Heartsmart" 
(e,g. group formation, seating arrangements)? iCi'^^cle one ) 

*Yes No 

*Please explain - ^ / 

Circle the appropriate response and/or comment. 

a. How well did the teacher's manual instruct or guide you in administer- 
ing the course and materials? (Circle one) 

*not very adequately well very 

well well 



*Please explain 



b. Was the teacher's manual organi-zed in such a way as to facilitate 
your understanding and implementation of the course? (Circle one) 

yes *No 

*Please explain • / 

■ > ' I 

\ 

I 
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c. If given ditto masters for "Heartsmart" student materials, would you be 
willing and/or able to reproduce them for distribution to your 
students? (Circle one. ) 



Yes No 
What I liked best about this course was 



What I didn't like about this course' was 
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■ C2a 

Student Attendance 



List students' names. 

At the end of each Lesson: 

• Check the 5 students who responded to an opinion survey. 

• i^lote "A" in the appropriate Lesson column for each student absent. 

Before the next Lesson: 

If students have made , up the work for the previous Lesson, (e.g» by 
listening to the tape and using the student materials by themselves 
in "free" time) circle the A previously noted. 
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STUDENT ATTENDAMCE 

SCHOOL # 



NAME 



LESSONS 
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. Student Opinion Surveys 

The student opinion surveys are intended to measure student reaction to the 
Heartsmart lessons r 

Survey A relates only to the tape-led lessons and is to be * administered after 
the completion of the taped game. 

Survey B relates only to the Heart-to-Hfeart talks and is to be administered at 
the. end of the discuss ioiT petiod. 

Data from both surveys will be used to make changes in the Heartsmart materials, 

Not all students will be asked to fill out forms after each lesson* Instruc- 
tions for selecting students to fill out the survey forms are given below. 

Directions for Administering Surveys • " 

1. Divide the class into groups of five, taking othe first five students on 
the class register as Group i, the next five students 33 Group 2, and so 
on. . — , — 

2. Immediately after Lesson 1, give copies of Survey A to the students in 
Group 1. Explain as follows: ; . 

The people who developed this program would like to know 
what you think of it. ^ lour opinion will help them to ■ ■ 
improve the program. 'Please answer the questions honestly. 
THIS IS NOT A TEST. , . • ^ 

Look at the ■ :iurvey._ Make sure you understand, the, questions^ 

For the last question^ you may write anything that you want 

to say. ,_ ' . 

Notice that the first two questions ai*e about tfie storzj' i 
■The last -three questions are about'' the game (activity) — 
the things you did right after the story finished. ; 

3. When the students have responded, collect the surveys* Please fill in the 
lesson number, the date, and your school code number. File these completed 
surveys in the back of this record book. 

4. Follow the same procedures for each tape-led lesson, rotating the groups . 
of students who respond -to the ouvvey. 
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After a Heart-to-Heart talk, the procedure is the same using Survey B. 
Group 1 is asked to fill out Survey B after the first talk, Group 2 
after the second talk, etc. 



After the last Heart-to-Heart talk — Lesson 29, ask all the students to 
respond to Survey B. 

Some copies of both surveys are included in this record book. Additional 
copies will be delivered to you by the RBS observer. 
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STUDENT OPINION SURVEY A. 
TAPE-LED LESSONS 



Date: 



Lesson f/ 



Code: 



1. How much did you like the story today? CHECK ONE 

□ A lot □ Some □ A little,- □ Not at all 

2. When you were listening to the story , what was your strongest feeling? 
'CHECK ONE OR^ wriiie in the space. 

I D Happy □ Sad □ Mad Other 



WHEN THE STORY ENDED, THE TAPE TOLD YOU HOW TO USE THE GAM2 PAGE, THINK 
ABOUT WHAT YOU DID TXJDAY. 



3. How much did you like the game "oday? CHECK ONE 

□ A lot I □ Some □ A little □ Not at all 



What did you t|hink about the game ? CHECK ONE 



□ It was f 



un 



Q It was not fun 



Was the game ed^y or hard? CHECK ONE 

□ It was easj\ D It was hard 



6. Use this space to\ write anything you want to say about the game today. 



ERIC 
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STUDENT OPINION SURVEY B Lesson // 

HEART TO HEART TALKS 

Date: Code: 



1. How much did you like the heart to heart talk? CHECK ONE 

□ a lot Dsome Qa little □ Not at all 

2. How much did ypu say? CHECK ONE 

n A lot Da little □ Nothing 

3. Did you want to talk more? CHECK ONE 

□ no DYes, a little DYes, a lot 

4. How interesting were the things the class talked about? CHECK ONE 

Very A little Not too , Not at all 

inceresting CD interesting CD interesting CD interesting 

5. What was the strongest feeling you had during the talk? CHECK ONE OR^ 
write iv the space. 

was happy I I I was sad 

□ i was mad was shy 

n I felt like laughing CD I felt embarrassed 

Other - ■ ^ ^ 



Use this space to write anything you want to say about this heart to 
heirt ta:.k. 
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STUDEIi* OPINION SUPA'EY C Code 

(Groups) 

J%il a 'V " in the I'ox to rJiou which ic your ancDcr. ^ 

1. Were you ever a leader? '^(Chcck one.) 

_ yes _ 

# 

If you put a in the (yes) box answer questions 2 and 3. 

If you put a in the (no) box ansv/er question 4. , 

2. How much did you like being leader? (Check one.) 

_ not at all _ some _ a lot 

3. Which was more fun -- being a leader or being a regular lobper? 
(CJiCck one. ) 

It v/as more fun being a leader. 

_ It v/as more fun being a regular looper. 

4. Would you have liked to have been leader? (Check one.) 

_ yes _ no 
Every oyie ansver the next four questions. 

5. How much did you like working by yourself? (Check one.) 

_ not at all _ some _ a lot 

6. How much did you like working in the "looper group"? (Check one 

. _ not at all _ some _ a lot 
Answer questions 7 and 8 by writing in your thoughts. 

7. What was the thing ycu liked best about your looper group? 



8. What was the thing you really disliked about your looper group? 
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HEARTSMART OBSERVATION 
(Lesson) 



)BSERVER: 
DATE: 
DAY: 
SCHOOL: 
LESSON #: 

PART I : CLASSROOM DATA 
NO. OF PUPILS 



NO. -/TYPE VISITORS: 



□ Parents 



TEACHER: □ 



□ 



School Staff 



EZlsubstitute 



npther 



ROOM ARRANGEMENT (e.g. seating pattern, direction P's face, "^'s desk, 

doors, special objects, etc.) 



Note Important change 
during lesson period 



PART II: LOG (Continued) 



SEGMENT 



OBSERVATIOhl' 



Activities (Start •) 



=?eview (Start ) 



/^rap-up (Start ) 



FINISH 



ERIC 
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PART i i : OBSERVATION LOG 



SEGMENT . : OBSERVATION 



Begin, (Start _) 



li ^^o. (Start ) 

i 



\ ■ 

I:; ^ory CStart ) 
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TIH Tape- led Lessons 
Observation Questions 

Were all materials prepared/set up/distributed for effective presentation 
of the lesson? YKS NO NA 



Where directed (manual or tape) did the teacher give additional instructions 
the students in a timely and effective manner? YES NO NA 



Did the teacher, on her own iniative, give additional instructions to the 
students in a timely and effective manner? YES NO 'iA 



Did the teacher circulate among the students? 

Did the teacher provide reinforcement/encouragement? YES NO NA 



Did the teacher use otlier materials/activities to supplement the lesson? YE; 



Did most students adequately demonstrate understanding of the taped 
directions? YES NO NA 



Did mo^t students appear to enjoy the lesson? YES NC NA 
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8. )ud :iny student (s) act or speak to another student or students in a way 
tiiat was not di reef i y called for by the less,pn, but which could be ^ 
attributed to the lesson? (e.g. hostility, empathy) YES NO NA 



'J. Did any student (not in interaction with others) exhibit stron^^^ feelings 
(e.g. discomfort, delight) not directly called for, but which could be 
attributed to the lesson? YES NO NA 



10. Was your overall reaction to the lesson favorable? YES NO NA 



11. Were tiiere any major problems with materials (including the tape)? YES NO NA 



12. Did the teaclier volunteer comments/anecdot:es related to recent lessons 
which were not observed? YES NO NA 



13. Do you liave recommendations for revisions to this lesson? (Please justify 
where possible,) YES NO NA 



JR:nls 
1-11-77 
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"HEARTSMART" OBSERVATION QUESTIONS 
(Heart- to-Heart Talk) 

TEACHER BEHAVIOR 

1. In -introducing the lesson did the teacher mention the "rules for qood 
discussion?" YES NO COMMENT: 



2. In introducing the lesson did the teacher talk about the meaning and 
importance of "good listening" during a H-H talk? YES NO COMMENT: 



3. Did the teacher explain beforehand what students were supposed to do 
during a H-H talk? YES NO COMMENT:.. 



4. Did the teacher tell students beforehand about what feelings they 
might have during a H-H talk? YES NO COMMENT: 



5. Did the teacher attempt to integrate previous lessons with this H-H 
talk? YES NO COMMENT: , " 



6. Did the teacher mention beforehand possible difficulties that stu- 
dents might have in tal king about their feelings? YES NO COMMENT: 



7. Did the teacher reinforce students' participation by repeating what 
students said or by reflecting their feelings? YES NO COMMENT 



8. Did the teacher share his/her own personal experience? YES NO COMMENT 



9. Did the teacher make all the main points and ask alj the questions on 
the "H-H Main Points and Questions Page?" YES . NO COMMENT: 



10. Did the teacher ask students new and different questions not listed 
on the ^'H-H Main Points and Questions Page?" YES NO COMMENT: 



STUDENT BEHAVIOR 

1. ' Was verbal interaction among students during the H-H talk relevant to 

the questions posed by the teacher? YES NO COMMENT: 

2. Did the students talk about their in-class peer relationships? 
YES NO COMMENT: 



S7o 



3. 



Did students talk about their peer relationships (in-class or out-of- 
class) in a variety of situations? YES NO COMMENT: 



4. Did any students show affirmative/positive feelings toward other mem- 
bers of the group? YES NO COMMENT: 

i 5. Did any students show angry or strongly negative feelings toward 
1 other members of the group? YES NO COMMENT: 

6. Did any students behave in ways which might suggest they were harmed 
by the H-H talk experience (e.g. crying, hitting, "escape")? 
YES NO COMMENT: 



7. VJas there any evidence of overt (non-verbal) expression of feeling 
toward members of the group (e.g. facial expressions, gesturing, 
physical contact)? YES NO COMMENT: 



8. - Did a majority of students participate (verbalize) during the H-H 
talk? YES NO COMMENT: 



9. Was most student talk directed toward the teacher? YES NO COMMENT: 



10. Was most student talk directed toward the group as a whole? 
YES NO COMMENT: 



11. Was most student talk directed toward individual members of the 
group? YES NO COMMENT: 

STUDENT OUTCOMES/CLASSROOM CLIMATE 

1. Did students appear to enjoy the H-H talk? YES " NO COMMENT: 



2. Did students relate their own personal experiences to what they "Wc- 'e 
learning from the "Heartsmart" program? YES NO COr^MENT: 

3. Did the class seem tense or uncomfortable/during the H-H talk (e.g. 

no smiling, no movement, long silences, /fidgeting in seats, aggressive 
behavior, crying, etc)? YES NO COMMENT: 

•. / ■ • 

/' 

/' 

4. Were there any interpersonal conflicts (between students) which did 
not get resolved by the end of the class period? YES NO COMMENT: 



\ 

\ 



t;h student interview 

!• Think about Jack and his adventures, and the things he learned. 
What did you learn from "Heartsmirt"? 



2. Do you tell your friends or parents or teachers your feelings more 
often now than you did before "Heartsmart"? 

CHyES (Go to 2A) ^ 



NO (Go to 3) 



2A. Tell me something about the feelings you show. What kinds of 
feelings do you show or tell about now? 



I • '■. ■ ■ 

i • ■ - ' (Go to 2B) 



^^ere do you show or tell your feelings now, that is, in what 
kinds of situations or places? t 



J (Go to 2C) 
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• / 

i ., /• 



2C. Would you tell me about one time or place that you sliowed your 

□ feelings? ^ 
YES .(Record ;i-esponse) : * 



CUnO (Go to 3) - ^ 



./ 

/ 

■/ 



3. Are you telling or showing your feelings fewer times now than before 
"Heartsmart" began? ^ , , 



FIyeS (Go to 3A) 
CZInO (Go to 4) 



3A. Tell me something about the feelings you don't show. What kinds 
of feelings don' t you show now? . 



(Go to 3E) 



^e^^- don't you show your feelings, that JLs,in what kinds of 
situations oi places do you not show your feelings? 



• ^ (Go. to. 3C) 

3C. Can you tell\nie why? 



(Go to4) 
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Do you think the children in your 'class have changec' since "Heartsmart" 
began? ' . f ' ' ' ' - ' " . . 

. CD YES (Go to 4A) 
Qno (Go to 5) 



4A. Tell me how they've changed. 






■■ ■ < 










\ 






(Go to 4B) 



4B. Do you like the way the class is .now, or did you like it better 
before '"Heartsmart" began? 



□ ' Now (Go to 4c) 

Before (Go to 4c) 



4C. Would you like to give m^e an example to explain? 
CZJyES /(Rscord' response) : 



iZjN0 (Go 'to 5) 
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5. When vou show your feelings now, does it usually help you? 
OyES (Go to 5A) 
NO (Go to 5C) 



5A, How does it help yon? 



(Go to 5B) 



5B. V/ould you tell me about a time when showing your feelings has 

□ helped you? 
YES (Record response) : 



I Ino 

(Go to 6) 



5C. Would you tell me how showing your feelings has not helped you? 



CGo^ to^) 
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6. Would you like to say anything else about "Heartsmart/' or do you 
have any questions you would like to ask? 

Record questions: 



Record . statements: 



Terminate Interview ' 



EKLC 
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School 
Code 



TIH TEACHER INTERVIEW 



Reactions of others to Heartsmart, 

1. Did you send the consent forms and information home with t he children 
OR did ^lou hand these, materials to the parents yourself? 

sent home with children (go to la) 

I handed directly to parents (go to Ic) 

la. Did any parents contact you before signing the consent form? 




Yes What were their concerns? How did you addres? these 
concerns? (go to lb) 




(go to lb) 



lb. Did j/ou contact any parents before or after they signed the form? 
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I I Yes How many? What were your reasons for doing this? 
What types of questions/concerns did they have? 
How did you respond? (go to 2) 



O No (go to 2) 



Ic Were the consent forms and information handed to the parents as 
a group (eg, after a PTO meeting) or individually? (go to lu) , 



Id What if any, questions or concet^ns did they have? How dij you 
respond? (go to 2) 
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Did any parents refuse consent? 

I j Yes How many? What were their reasons for refusing? (go to 3) 



I I No (go to 3) 



Of the parents who gave their consent, did any later wish to withdraw 
their child from the course? 

I I Yes What were their reasons for wanting to withdraw the child. 
' How did you react or respond? Did the parents change their mind 
or was the child withdrawn? (go to 4) 
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[~\ No (go to 4) 
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, 7. ' ■. nvite parents to observe lessons? 

j ' '^-i What kind of response did you get to. the invitation? 

If any parents did observe, what was thair reaction(s)? 
(go to 5) 



13] No (go to 5) 



f,. Did >..'t. "re ort "back" (or are you planning to "report back") to parents 
at the end the course? 

H Q Yes (r.3k for -K.formal explanation.) (go to 6) 
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I I No (go to 6) 



Have you discusse'^ the program with the other teachers in your school? 
(Ask for infonna'i e.xplanation. ) . 



□ Yes (go to 7) 



r~| Ho (go to"7) 



Diu you discuss the program wi''h anyone else? (Ask for informal expla 
iidtiiOn, ) ■ 6' 
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I I Yes (go to 8) 



I I No (go to 8) 



d you show pro gram materials to other 
Q Yes Which materials? Any of 

vities? . 
What wer^. their reactions? 

(go to B-9) 



teach<^rs, friends, or colleagues 
those used for the optional acti 
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I I No (go to B-#9) 



B. Experiences with other Programs. 

9. Have you used or heard about other programs that are similar in content 
and/or objectives to Heartsmart? 

[~] Yes What are they? 

How have they compared to Heartsmart? 

(Probe: In terms of.,- appeal, style, target group, materials, 
administration, effectiveness), (go to 10} - 



I I No (go to 10) 

. 1 
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10. What future do you see for the Heartsmart materials? 

(Your district. others?) 

Probes: How should the program be used? 

How should it be integrated into the existing curriculum? 
Do you feel that the program is important enough to justify 
taking away time from other subjects? (go to 11) 



11, How^ effective or ineffective did you find the format of the Heartsmart 
program? (i.e. tapes, storybook pages to color, group folders etc-) 
How pleased were you with the format? {go to 11a) 
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lla. Was the organization and contefnt of the Teachers Manual to your 
liking? ^ 

I I Yes What did you like best about the manual? (go to lib) 



I I No Please describe emy changes that would make the organi- 
zation or content of the manual more to your liking* 
(go to lib) 



lib* Which parts of the" program (with respect to fomiat) seemed 
important, and^which seemed superfluous (go to lla) 
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nc. Do you have any suggestions for ways the program could be pack- 
aged so it would be less bulky or e asier to administer or less 
expensive? 

rn Yes (go to 12) . 



/ 



n No (go to 12) 



12a. Has anything in your experience, training. or out-of-school acti- 
vities facilitated your teaching of the Qburse? 

'V Yes (Please explain) / - 



□ No 



12b. Has anything in the school — the general^ atmosphere, other 
, courses, the school philosophy — facilitated your teaching of 
the course? . . 

□ Yes. ■ ^ ' 



f 



□ No 



/ 



^ / 
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12c. Has "Heartsmart" had any effect on you? (Have your behaviors, 
teaching style or attitudes to yourself, to students- or to other 
adults changed?). ' ' ■ 



Yes 



□ No 



Student Reactions. 



■ / 



13. How do you think::_your students felt about " the. prdgrani?~~TProbes: Did 
they Took forward to it? Did they talk about it to you? With^^ne • 
' ; another?) (go to 14) . 



\ 
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14. Did the students behave differently as a result of the Heartsmart 
materials? 

I I Yes (go to 15) ^ . 



. Q No (go to 15) 



15. Are there certain students in your class, who seemed to benefit 'a lot 
from" the program? :1 ' 

Yes Why did the prograCseem to work so well with these 
children? (go to 16) 



ERIC 
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I I No (go to 16) 



16. Were there any children in your class who you feel should not have 
participatedHn the progrem? 

L] Yes (go to 17) 



n No (go to 17) 



17. From your experiences, what were some of the benefits, if any, that 
accrued from the Heartsmart program? (go to 18) 
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18. What if any were some of the disadvantages (discomforts) that resulted 
from the Heartsmart instruction? (go to 18a) 



18a. Was there much negative peer reaction between students? 
I I Yes Could you have used more advice? (go to D-19) 



I I No (go to D-19) 



ERIC 
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D. Use of Heartsmart Concepts 

19.- Did you have any opportuni tites to use Heartsmart-related , 
vocabulary, or activities with your class 'during other cias i . 
activities? 

I I Yes Explain/Describe (go to 20) 



|~] ^ (go to 20) 



20. Refer to the teacher's log that lists the optional activities com- 
pleted. Ask teacher to expand the explanation of any he/she developed 
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20a. How ,helpful and necessary did you find the optional activities? 
When, in your opinion, are they most helpful Jbest used)? 



21. Did you choose to make any changes in your schedule, in the way you 
administered the program, or in the amount of time you spent on 
prog ram materials? 

I I Yes Please describe/explain, (go to 23) 



r~] No (go to F-24) 



Siimmary 

*22. Refer to the teacher's Unit Opinion Surveys and ask him/her to discuss 
or expand his/her responses, (go to 23) 
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*23. Invite teacher^to ask you any questions, and/or to sumnarize his/her. 
overall reactions to the program and field test experience. 
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WHAT WOULD YOU DO? . 

This is not a test. We would like to know what you would do in each of these 
situations. Think about the situation and then choose the answer that you 
think tells what ^ would do. Put a check (/) next to your choice. 

1. Everyone tells Mart.n how wonderful he was in the school play. He feels 
embarrassed by what they say. If you were Martin, what would you do? 

_ (a) Say you're embarrassed, but thank them for the compliments. ; 

_/(b) Try to change the subject- 

Say, "I wasn't really so good." 

I. Ellen visits her friend Jane, but Jane doesn't want to play. This hurts 
Ellen's feelings. If you Were Ellen, what would you do'? 

(3) Think that Jane is angry with you for something, and then walk away. 

(b) Tell her that your feelings are hurt, and then ask her to explain 

why she doesn't want to play. 

_ (c) Tell Jane you'll see her some other time, and walk away. 

3. Dor^othy and Bonnie are playing ball when Dorothy's little sister starts 
.bothering them and getting in the way of their game. If you were Dorothy, 
what would you do? 

(a) Tell your parents that your little sister is being a pest and to 

make her go away. 

(b) Tell your little sister to go somewhere else to play. 

(c) Tell your little sister that you feel" angry at her, and ask* Ker to 

find something else to dp. 

4. Five boys and girls in your class start a secret club. You want to'be in 
the club, .but the other kids haven't asked you. What would you do? 

_(a) Tell the kids that you feel left out, and that you would- 1 ike to be 
in their club. 

(b) Think that the other kids don't like you and feeT angry at them. 

(c) Tell your teacher that the other kids are starting^ a secret club 

and to make them let you join. 
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You feel very .sad because your parakeet died. What would you-^o? 

(a) Ask your parents to get a new parakeet. 

(b) Think that it was ^only a bird and not very important. 

(c) Be sad and even cry if you feel like it. 

You want to watch your favorite show on TV but your older sister ,'^^nts- to 
watch^ another show. She pushes you away and puts her show on. Wiiat would 
you do? ' " - . 

(a) Get your mother or father to make your sister change the channel. 

( b) Tell your sister you really want to see your show a lot and ask her 

to let you watch it. . ^ ■ 

( c) Push your sister back and switch , on your show.; 

Your cousin won't let you play in his new treehouse. What would you do? 

(^) Ask him why he won't let you play in the' treehouse , ,and tell him 
' how you feel about it. 

( ^) Go into the treehouse when he's not there 

(c) Ask his mother to let you play in the treehouse. 

Donald gets mad when he sees his little brother's toys all over his half, 
of the bedroom. ^.If you were Donald, wha"" would you do? 

(a) Beat up your brother, and make ^^m move his toys. 

(b) Tell your brother you don't like the toys, on your side and ask him 

to move them. 

(c) Put your brother's toys, and some of yours, all on his side to teach 

him a lesson . 

Last week Amy and Nora had a fight. Nora went home from Amy's crying, 
even though she thought she was right. Now Amy asks Nora to come over to 
play. If you were. Nora, what would you do? 

(a) Tell Amy how you felt about +ne fight, and Why you felt that way. 

( b) Sa.y, "I'm sorry, but I can't play with you today." 

( c) Go over to Amy's and pretend there was no finht. - ' 
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It was V'alentines Day. ■ Sarah sent Judy a valentine but Judy didn't send 
one to Sarah. Sarafi's feelings are hurt. If you were Sarah, wh^t would 
you do? • . * , 

^ (^) Ignore Judy for awhi le , even though you. still want to be her friend 

— (ta) Ask Judy why she. didn't send you a. valentine, and expL.in that your 

• feelings wer?€ hurt. 

(c) Think that Judy doesn't like you any more. j , 

Jakeo doesn't feel like going to Cub Scouts because he wants to be alone. 
His mother asked him why he isn't going. If you were Takeo what would 
you do? c . • 

_(a) Tell her you think you-' re getting sick* and want to stay home. 

• (h) Tell her you feel like being alone 'today. ^ 

(^) Say that you've changed your mind and will go to Cub Scouts. 

John and Tim are picking^ on Bot every day at school and teasing him a lot. 
If you were Bob what would you do? 

(3) Tell them that, their teasing upsets you. 

( b) Ignore them and walk away. . 

— (c) Ask the teacher to talk to John and Tim. 

Matthew is sick and his father is taking him to the doctor. Matthew is 
scared that he's going to have a shot. If you 'were Matthew, what would 
you do? 

(3) Act brave and not let anyone know you're afraid. 

^ (b) Tell your father you don't. need a shot. 

(c) Tell your father and ^he doctor that you'-re scared ot having a shot. 
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At her birthday party Humiko doesn't like -it because everyone is playing 
with the new toys and not paying attention to her. If you were Humiko, 
what would you do? 

(a) Tell them you want'them to play more with you. ' . 

(b) Start acting mean to the kids. " 

(c) Ask her mother to put the toys in another room. ' 

Fred did very well when he was reading out loud in his «group. Everyone 
enjoyed his reading and he felt glad.' If you were Fred, what would >;ou d 

(a) Feel qlad inside, and keep your feelings to yourself. 

_ (b)\Tell them it's easy to read well. 

(cj Let them know you feel glad. ' ^ 

All of Billy's friends were going to ttie movies on Saturday, but' they 

didn't invite him to. come along. Billy's feelings are hurt. If you were 
Billy, what would you do? 

(a) Stay home and do something else, and try to forget about it. 

(b) Tell them how you feel, and ask them if you can yo, too. 

(c) Show up at the movies at the same time they d"o. 

Jane has been waiting her turn to get a drink of water and a boy in her 
class cuts in line in front of her. If you were Jane, what would you do? 

(a) Push him out of the way. 

: (^) Tell him you don't like what he's doing, and, ask him to wait his 

turn. 

, (c) Ignore him, because your turn wll"^ come prfetty quickly anyway. 

Richard, who is new in the scf?bol^, doesn't have any friends. Everyone 
ignores him. If you were Richard what would you do? ' ' 

(a) Ask someone you like to play with you. 

(b) Plav bv yourself. >^ 

' (c) Try to take charge. 
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WHAT WOULD YOU DO? 



This IS not a test. We wou^d^^ike to know what you would do in each of these 
situations. Think about the situation and then choose'-the answer that you 
think tells what ^ would do. ^Put a check (/) next to your choice.- 

i 

1. Everyone tells Ma>^tin how wonderful he was*in the school play. He-feels 
embarrassed by. what they say. If you were Martin, whatwouTd you do? ■ 

; _^ (a) Say you're embarrassed, but thank them for the compliments. - 

■ ( b) Try to change the subject. , 

_ (c) Say, "I wasn't really so good." 

2. Ellen-visits her friend'' Jane , but dane doesn't want to play. This hurts 
Ellen's feelings. If you were Ellen, what would you do? c; 

_ (a) -Think that Jane, is angry with you for something, and 'then walk' away. 

(b) Tell her that your feelings are hurt, and th'en asK her to ex-plain 

why she doesn't want to play. ' ■ ^ 

(c) Tell Jane you'll see her some other time, and walk away. 

3. Dorothy and -Bonnie are playing ball when Dorothy's little sister starts 
bothering them and getting in the way of their game. If you were Dorothy 
what would you do? . . • • 

(a). Tell your parents that your little sister is' being a pest and to 

make her go away. " . ^ .. 

(b) Tell yourjittle sister to go somewhere else to play. 

vc) Tell your ^little sister that you feel' angry at Jief, and ask her to 

find something else to do. ' 

4. Five boys and girls in your class start a secret club. You want to be in 
• ^ the club., but ..the other kids haven't -asked you. What would you do? 

_ (a) Tell the kids; that you feel left out, and that you would like to be 
their club. ' . 

(b)Jhink.. that the other kids don't like .you and- f^e.l angry at them. 

_ (c) Tell your teacher that the other kids are starting a secret club 
and to make ^them let you join. 

■1 
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5. You feel very sad because your parakeet diec'. What would you do? 
(a) A^k your parents to get .a new parakeet. 

(b) Think that it was only a b.ird and not very important. 

( c) Be s-ad and even cry if you feel like it r 

6. You want to watch your favorite show on TV but your older sister wants to . 
watch another show.. She pushes you away and puts her show on. What would 
you do? 

_ ( a) Get your mother or father ^o make your sister change the channel. 

(b) Tell your sister you really want to see your show a lot ttnd ask her 
to let you watch it. ^ 

( c) Push your sister back and switch on your show. 

7. Your cousin won't let you play in his new treehouse. What would you do? 

(^) As^k him why he won't let you play in the treehouse, and tell him 
■ how you feel about it. ' " - ' 

( ^) Go into the treehouse when he's not there-. 

(c) Ask his mot+ierr^to Tet you play in the treehouse. 

* . * 

8;. Donald gets mad when he sees his^s little brother's toys all over his half 
^ of the bedroom. If you were iDonald, what would you do? 

(a) Beat up your brother, and make him move his toys. 

(b) TelT your brother you don't like the toys on your side and ask him 

to move them. 

( c) Put your brother toys, and ?ome of yours, all'- on hisXside to teach 

. • hjm a lesson. , ^-^ \ 

9. Last week Amy and Nora had a fight. Nora went home from Amy's' crying, 
even though^ she thought she was right. Now Amy asks Nora toxcome \over to. 
play. If you were Nora, what would you da? , _ , " . \ 

( a) TeH Amy how yDtr""felt about the fight, and why you felt that>way. 

( b) Say, "I'm sorry, but I can't prlay with you today." y 

^ ( c) Go over to Amy's 'and pretend .there was no fight.. 
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It was Valentines Day. Sarah sent Judy a valentine but Judy didn't send 
one to Sarah. Sarah's feelings are hurt. If you were Sarah, what would 
you do? 

.(3) Ignore Judy for awhile, even though you still want to be her friend. 

: — ( b) Ask^Judy why she didn't send you a valentine, and explain that your 
feelings were hurt. 

Tc) Thfnk that Judy doesn't like you any more. 

Takeo dqesn't feel like going to Cub Scouts because, he wants to be alone. 
His mother asked him. why he. isn't going. Jf you were Takeo what would 
you do? . 

(3) Tell her you think you're getting sick, and want to stay home. 

( b) Tell her you feel like being alone today. 

(c) Say that you've changed your mind and win go to Cub Scouts. 

'John and Tim are picking on Bob every day at schpol^and teasing him a lot. 
If you were Bob what would you do? 

(3) Tell them that their teasing upsets you. 

( b) Ignore them and walk away. 

_^(c) Ask the teacher to talk to John and Tim. 

Matthew is sick and his father is taking him to the doctor. Matthew is 
scared that he's going to have a shot. If you were Matthew, what would 
you do? ~ 

- 

(a) Act brave and not let anyone know you're afraid. 

(b)'TeH your father you don't need a shot. 

(c) Tell your father and the doctor that you're scared of having a 'shot. 
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14. P^t her birthday party Humiko doesn't like it because everyone is playing 
wi\.n the new toys and not paying attention to her. If you were Humiko, 
what would~you do? 

(a) Tell them you want them to play more with you. 

( b) Start acting mean to the kids. ' ' . ^ 

""fc)" Ask -her mother to-put^-the toys- in-arrother room. ■ j 

15. Fred did very well when he was reading out loud in his group. Everyone! ^ 
enjoyed his reading and he felt glad. If you were Fred, what would you do? 

(a) Feel qlad inside, and keep your feelings to yourself. 

( b) Tell them it's easy to read well'^^ 

( c) Let them know you feel glad. 

16. All of Billy's friends were going to the movies on 
didn't invite him to come along. Billy's feeTings 
Billy, what would you do? 

(a) Stay home and do something else, and try to 

(b) Tell them how you feel, and ask them if you 

(c) Show up at the movies at the same time they 

17. ^Jane has been waiting her , turn to get a drink of water and a boy in her 

class cuts in line in front of her..' If you were Jane, what would you do? 

(a) Push him out of the way. 

_ (b) Tell him you don't like what he's doing, and. ask him to^wait his 
turn . ' 

(c) Ignore him, bec/ause your turn will come pretty quickly anyway. 

18. Richard, who is'pew^in the school, doesn't have any friends. Everyone 
ignores him. If you were Richard what WQuld you do? 

('a) Ask someone you like to play with you. 

(b) Play by yourself. 
(c) Try to* take charge. ^ . • 



Saturday, but they 
are hurt. If you .were 

forget about it. 
can go, too. 
do. 
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'VV)i:>t Would Do?" Si'o.liia K(.-v 



2. 



Al* 
IJ4 

A 5 

C4 

Ad 
R7 

A3* 
B.V 
C6 



6 . A6 
C7 

1 . A 2* 

B7 
Cb 

8. A7 

C7 

9. Al* 

B4' 
C4 



11 



12. 



A4 • 

Bl* 

C4 

B4 
06 



13. A4 

cr-'- 

14. A3* 

li7 

' ' r.6 



15. 



16. 



17. 



18. 



A4 
B8 
C)'= 

AA 

B3" 

C8 

A7 

B3* 

C4 

A3* 

B4 

C8 



A-(. 
C1-- 



10. 



A4 

B2* 

C5 



KEY 

I. Dc^sired Program-Related Responses 

1. Expression of • feelings 

2. Asking others to express feelings, and stating own feel- 
ings 

3. Expressing feelings/desires and requesting action of 
another to facilitate "desire. 

II. Unacceptable Program-Related Responsei: 

4. Hiding feelings 

5. Guessing other's feelings 

G. Uepending on authority/others 

7. Takir.g physical action against another person or taking 
action which negatively affects arothar person 

II [. Unrclatcii Program Responses 

8. Other (catch-all category) 

inJicatcin "correct" response 
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WHAT*S HAPPENING? 

This is not a test. There are eight (8) sets of cartoons. There are questions 
about each set of cartoons. Try to answer the questions as well as you can. 
Don't worry about spelling. You may write one word answers if you like/ Turn 
*he page only when you are told that you may do so. 
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1. What feeling did the girl show in the three pictures? 



2. Why did she do the things she did in the three pictures? 




1 



In picture #2, Tony is showing a feeling. What feeling is he showing? 



Tony's sister thinks he is angry. What should she do to find out if she 
is right? ' — — 

Why do you think Tony is acting the way he is in picture J4? 



How do you think Tony really feels about being in charge when his mother's 
not in the house? 



GRANDMA VISITS THE BROWN FAMILY 




How to you think grandma . feel s . about the Brown family in the first three 
pi ctures? ' — — : 



2. Tell one thing grandma did to show her feelings. 



3. a Do you think that Joe wanted to kiss grandma? 

CIRCLE ONE: YES NO I DON'T KNOW 

b . Why or why not? — — . 

4. If grandma v;ants an answer to the question she's thinking in picture #4, 
what should she do? 
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1. What happened to make the team take a vote? 



2. What was Sally feeling when she said "Only one for me?" 



I 3. Draw a circle around the words Tim said that tell how he felt about being 
captain of the team. 

4 
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is telling the truth? 



CIRCLE ONE: ' YES NO 
b. If you circled YES, explain how you could tell. 



c. If you circled NO, explain how you could find out the truth. 



2. How ^an you tell how Mike feels in picture #4? Write as many answers as 
yoLi knowx. 
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1. Why did; Cindy act the way. she did in the first three pictures? 



2. Why do you think Cindy is acting the way she is in picture #4? 



6jl0 




1. What was the boy feeling in the story? 



2. What 'happened to make the boy feel that myl 



3. Why did th'e boy act the way he did in picture //2? 
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Al. •Uai>iyinci;s/crfcci:ion7ri:iend.liue3s 

A 2. She was hai';py . /Sl^e v;as feeling friendly. ' 

l> ^ Auv=c-r He 4oer.n'l: likri being in 

charge. /He. 'is scared- about 
n 2... She BhouLI p.rA: Tony v/hal: hc-^ bc-'Snz charge. . 

1)3. Tony is scared/v;orvjed. 

C 1. She 1x1: is tiheia/fcels affectionate (cuid vants them to J.ike her) . ^ 

•C2. Any o;K-r of Lhc following: ^ : 

e She v:anted to kl?s Jca:. " She h^o.de a. cake for thciu. 
1^ She ^.-^id they looked well. • She af;ked al)Out their vacation.. 

o She s\ailed aU them. ^ . ^ 

C 3. a. anrl h. ,cire velatcd. . . , .^^^ ^ »v 

.JJT circ.Ie5; /VO, he or sh<^ should explain that Joe doesn^^ t 

feci like kl::.r>ing just now. ' - ' j,' ,^ 

If student circles YES, he or 'she , should explain that Joe does n t 
want to hurt griujdr.a's feelings. ' . ^ 

CA. She should go hack cuid ask them. • . 

D 1 . Sally said Tim w.is no^ good . D 3. St .dent should circle Tim's 
Sally said they should vote. words ■ in^picture n. .. 

D'2. Sad/cai;',ry/disrppc)jnted 

I'. !• ^l. r h. and «hj c related, 

a. No. 

b. Student should not respond. 

c. Acik Joe why Hike was not invited. 

E 2. Student may be scored on the number of valid response.'^ made. Any 
or c'ill of tne following are appropriate: 
^ e He says, 'No one likes me." o He is crying. 
o lie is" alone. ^ His fcce is sad. 

e His body Looks sad. 

V 1. She wanted someone, to play with. 
F 2. She ±r. lonely/has . no one to play with. 

_ — — : ' 
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p I, s;{:iu:i-:ii^s ;;; ?./ bo- :^corocl ou LJu^ j^ihr-r^-r of valid rc:Spon:ja:: xhrL 
Arnj 02^' all i'rio follcrjiivj rjr^^ C'j.n.vopryntc : 
"c. Shrv ^aid,- »'0h Joy»/' "Wiroopee I ■ cad "I m;Kkr the te£J:n." 
6 Sl;e.3> s'i.jV.I (shov:r. f coIiii.:;;-s uJ th her fcco) • 

o 5h':i cioes a 'carlvljae.l.l (s;hcr.;s fcc?.l.Lnc;s widi har body) 
fi*" She v/avcs l>oi: hrnidr.^ 1 . 

She kisses lier falhc^r (st^hrci: feelings). 

1-1 1 • A'ilgxy • ■ ' ' , 

112. He h^d too miu-u horr»ov*oilv/ couldn ' t play ba5>eball. 
H 3. He wns angry/feeling i.iad/in a bad mood.. 



Proj^rhiu Objectives 



^ Identify (infer) feelingr* rmd 
behaviors c\pd. infer feclingr; ... 
- frora behavior :ih otliersl 

o Identify Uciys in.\;bic.h pc^bple 
may behave T>7hen the3'^ have 
certain ^:eelings» . !. ^ 

a Identify Cc-.u^e mid effect 

relationships between feelings 
0X1(1 beh^^^ior. .' • ' 

• DetermiviC' feelings in others 
by askin/^ (not guessing) . 



Related Items ^ 



A] , Bl, B^i, CI, B2, HI. 

C2, D3, E2, Gl. . 

A2, B3, C3, m,. Fr, .F2, 
H2, 1)3. 

B2, -C4, Ela, Elc. 
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Circle 

Your 
Answer 



1. The children in this class enjoy their schoolwork. ^ Yes No 

2. Children are often fighting with each other. Yes No 

3. My be3t friends are in my class. Yes No 

4. Some of the children in our class are mean. Yes No 

5. Most childr^en are pleased with the class. Yes No 

6. Many.^c.hildren in the class play together after 

school. Yes No 

7. Some children don't like the class. Yes No 

8. Many children in our class like to fight. Yes No 

9. -In my class everybody is my friend. Yes No 

10. Most of the children in my class enjoy school. Yes No 

11. Some children don't l ike o ther children. . ' Yes No 

12. In my c.^ass I like to work with others. ^ Yes » No. 

13. Most children say the class is fun. . ' Yes No 

^14. Some children in my class are not my friends. Yes No 

15. Children have secrets with other children in 

the class., ' ^ Yes ^ No 

"16. Some children don't like other children. Yeg No 

17. Jsome children are not happy in class. Yes No 

J 

18. All of the children know each other well. Yes ^No 

19. Children seem to like the class. Yes No 



20. Certain children always want to have their own 
.. way. 



Yes No 

21. All children in my class are close friends. Yes No 



22. Some of the children don't like the class. Yes No 
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23. Children in our class fight a lot. 

24. All of the children in my class like one 
another. 

25. Certain children don't like what other children do. 

26. The class is fun. : 

27. Children in our class like each other as friends. 



Circle . 
Your 
Answer 

Yes No 



Yes 
Yes 
Yes 
Yes 



No 
No 
No 
No 
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Please tell how you really feel or act. 
There are no right or wrong answers. 
Take as much time as you need to finish 

Part I — What to do : 

Each time read the sentence. Read the groups of words under the sentence. 
Choose the ONE word or group of words you want to put in the gap. 
Draw, a line under the answer you choose. 

ONLY DRAW A LINE UNDER ONE WORD OR GROUP. OF WORDS FOR EACH SENTENCE. . 

1. I try to make children do what I want them to do . 

most many some a few one or two no (none)^ 

2. I try to be very friendly and to tell my secrets to children. 

most many some a few one or two no (none) 

3. I like children to invite me to take part in what they^re doing 

most many some a few one or two no (none) 

4. What I do depends a Idt on what , children tell me. 

most many some a few one or two no (none) 

5. I like'' children to act as if they don't know me very well. 

most many some a few one or two no (none) 

6. I try to take charge of things when I am with children. 

most many some a few one or two no (none) 

7. I act unfriendly with children. 

most many some a few one or two no (none) 

8... I like children to ask me to take part when they're talking 

aiyou;t something. 

most many some a few one or two no (none) 

9. I let. . children take charge of things.. 

most many some a few one or two no (none) 



1 
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10. I like children to act friendly to me, 

most many some a few one or two no (none) 

II . I try to have children do things the way I want them done. 

most many some a few one or two 

12. I don't get very friendly with children. 

most many some a few one or two 

13. I like children to chooae me for a friend. 

0 

most many some a few one or two 

14. I take orders from children. 

most many some a few one or two 

15. I like children to act very friendly to me 

most many some a few one or two 

16. I try to be friendly to children. 

most ~ many some a few one or two 

17. I like children to ask me to join in what they're doing. 

most many some a few one or two • no (none) 

18. I follow what children are doing. 

most many some . a few one or two no (none) 

i 

119. I like children to get to know me very well. 

i ' ' ■ ^ ^ 

j most many some a few one or two no (none) 

20. I try to have close, warm friendships with children. 

1 most ' many some a few one or two no (none) 

i ■ 

21. I let children tell me what to do. 

! most many some a few one or two no (none 

I" i • ■318 • 



no (none) 



no (none) 



no (none) 



no (none) 



no (none) 



no (none) 



22. I<like 



children to act very friendly and tell me their secrets. 



most 



many 



some 



a few 



one or two 



no (none) 



Part II — What to do : 

Each time, read the sentence. Read the groups of words under the sentence, 
Choose- the ONE word or group of words you want to add to the sentence. 
Draw a line under the answer you choose. 

ONLY DRAW A LINE UNDER ONE WORD OR GROUP OF WORDS FOR EACH SENTENCE. 
^^23. When other children are playing games, I like to join them . 



almost all 
the time 



a lot of 
the time 



sometimes 



once in 
a while 



almost 
never 



never 



24. I xry to take charge of things when I'm with children 

sometimes ^ 



almost all 
the time 



a lot of 
the time 



once in 
a while 



almost 
never 



never 



25. I try to have close, warm friendships with children 

sometimes 



almost all 
the time 



a lot of 
the time 



once in 
a while 



almost 
never 



never 



26. I like other children to invite me to their houses when they are having 
friends over . 



almost all 
the time 



a lot of 
the time 



somet5.mes 



once in 
a while 



almost 
never 



never 



27. What I depends a lot oh what other children tell me 

sometimes 



almost all 
the time 



a lot of 
the time 



once in 
a while 



almost 
never 



never 



28. I like children to act as if they don't know me very well 



almost all 
the time 



a lot of 
>)the time 



sometimes 



once in 
a while 



almost 
never 



never 



29. . When a group of children gets together to do something, I like to Join 
in with them . • 



almost all 
the time 



a lof bf 
the time 



sometimes 



once in 
a while 



almost 
never 



never 



30. I" try to make other children do what I want them to do 
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almost all 
the time 



a lot of 
the time 



sometimes 



once in 
a while 



almost 
never 
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never 



31. I ti-y to have close friendships with children 



32. 



almost all 
the time 



a lot of 
the Lime* 



somet Imes 



1 Like to be invited to parties 



almost all 
the time 



a lot of 
the time 



sometimes 



once in 
a while 



once in 
a while 



almost 
never 



almost 
never 



never 



never 



33. I take orders from other children 



almost all 
the time 



a lot of 
the time 



sometimes 



once in 
a while 



almost 
never 



never 



34 . T like .children to act very friendly to me 

sometimes 



almost ail 
the time 



a lot of 
fhe time 



once in 
a while 



almost 
never 



never 



35. I try to take part in clubs and school-groups 

sometimes 



almost all 
the time 



a lot of 
the time 



once in 
a while 



almost 
never 



never 



36. I like to tell other children what to do 

sometimes 



almost all 
the time 



a lot of 
the time 



once in 
a while 



almost 
never 



never 



37. I try to have friends that I can be very friendly with and ttll my 
secrets to . 



almost all 
the time 



a lot of 
the time 



sometimes 



once in 
a while 



almost 
never 



never 



38. I like children to ask me to loin in what they're doing 

sometimes 



almost all 
the time 



a lot of 
the time 



once in 
a while 



almost 
never 



never 



39. I let other children tell me what to do 

sometimes 



almost all 
the time 



a lot of 
the time 



once in 
a while 



almost 
never 



never 



40. I like children to act not too friendly to me 



almost all 
the time 



a lot of 
the time 



sometimes 



once in 
a while 



almost 
never 



never 
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. I ^try to include other children in my plans 



almost all 
the time 



a lot of 
the time 



sometimes 



once in 
a while 



alu^ost 
never 



never 



42. 



I try to be the boss when I am with other children 

sometimes 



almost all 
the time 



n Jot of 
the time 



once in 
a while 



almost 
never 



never 



43- 



44 



I try to get very friendly with other children and to tell them my 
secrets . ^ 



almost all 
the time 



a lot p.f 
the time 



sometimes 



I like children* to invite me to things 

sometimes 



almost all 
the time 



a lot of 
the time 



once in 
a while 



once in 
a while 



almost 
never 



almost 
never 



never 



never 



45. 



46. 



47, 



I follow what other children are doing 

sometimes 



almost all 
the time 



a lot of 
the time 



once in 
a while 



I like other children to get to know me very well 



almost all 
the time 



a lot of 
the time 



sometimes 



once in 
a while 



almost 
never 



almost 
never 



When children are doing things together, I like to join them 



almost all 
the time 



a lot of 
the time 



sometimes 



once in 
a while 



almost 
never 



never 



never 



never 



48, 



49, 



I try to have other children do things I want done 

sometimes 



almost all 
the time 



a lot of 
the time 



once in 
a while 



almost 
never 



never 



When I'm going to do something I try to ask other children to do it 
with me 



almost all 
the time 



a lot of 
-the time 



sometimes 



once in 
a while 



almost 
never 



never. 



50. I like other children to choose me for a friend 

sometimes 



almost all 
the time 



a lot of 
the time 



once in 
a while 



almost 
never 



never 
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51 • I like to go to parties 



almost all 
the time 



a lot of 
the time 



sometimes 



once in 
a while 



almost 
never 



never 



52* I try to have other children do things the way I want them done 



almost all 
the time 



a lot of 
the time 



sometimes 



once in 
a while 



almost 
never 



never 



53. I try to have other children around me 

sometimes 



almost all 
the time 



a lot of 
the time 



once in 
a while 



almost 
never 



never 



54. I join clubs 

almost all 
the time 



a lot of 
the time 



sometimes 



once in 
a while 



almost 
never 



never 
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APPENDIX D 
EVALUATION DATA 

1. Correlation between total scores on What's Happening?, 
and What Would You Do? tests-. 

2. Intercorrelations among selected My Class Inventory Scales. 

3. ^nter correlations among FIRO-BC Scale's. ^ _ 
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Correlation Between Total Scores on. 
What's Happening? And What Would You Do? Tests 



Dl 



What's 
Happening? 



What Would 
You Do? 




P ^ . 01 
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Intercorrelations Amoi>g Selected 
My Class Inventory ^Scales^ 



SCALE 


Satisfaction 


Friction 


Cohesion 


Satisfaction 




-.48* 


• 37* 


Friction 


-.45* 




-.32* 


Cohesion 


.38* 


-.45* 





Pre-Test correlations are below diagonal; 
Post-Test correlations above diagonal. 
Sample N=359 



P < .001 
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Intercorrelations Among FIRO-BC Scales 



SCALE 


Expressed 
Inclusion 


Wanted 
Inclusion 


Expressed 
Control 


Wanted 
Control 


Expressed 
Affection 


Wanted 
Affection 


Expressed 
Inclusion 




.65* 


.02 


.27* 


.45* 


.42* 


Wanted 
Inclusion 


.58* 




.09 


.27* 


.45* 


.54* 


Expressed 
Control 


.06 


.08 




.32* 


-.25* 


-.11 


Wanted 
Control 


. .34* ., 


.31* 


.34* 




.16 


.12 


Expressed 
Affection 


.40* 


.39* 


-.28* 


.07 




.54* 


Wanted 
Affection- 


.26* 


.40* 


-.29* 


-.06 


.57* 





Pre-Test. correlations are below diagonal; Post-Test correlations above diagonal 
Sample N=359 - 



P < .001 
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